One of the many things I haven’t really thought much about before is how people test websites these days. I know how we used to do it in the early days of the public-facing internet (when dinosaurs roamed the Earth). This was long before things like AI-assisted vibe coding were a twinkle in anyone’s eye.

Basically, someone would code the website by hand, with assistance from graphic artists and content creators. Then everyone would merrily prod the sandboxed version—pressing every button, clicking every link, and generally trying to break everything. Eventually the website went live, at which point users promptly discovered all the bugs we’d missed. Ah yes… happy days.

Things were hard enough with the rudimentary websites of the time: lots of text was coupled with a limited number of fixed graphics, there were only a few browsers to contend with, and users interacted with websites only on large screens on personal and workstation-class computers.

Today’s websites are completely different beasts. They need to work on multiple operating systems, multiple browsers, multiple browser versions, multiple screen sizes, and multiple device types ranging from smartphones and tablets to desktop workstations. How on Earth do you even begin testing something like that?

But wait, there’s more, because many of today’s websites feature overly talkative AI-powered chatbots. If you are using one or more of these bots on your own website, how do you know that it’s helping your customers and not insulting them or pointing them at your competitors, for example?

And while we’re on the topic of chatbots, what about the ones that answer telephone calls from your customers, patiently listening to their seemingly endless questions and complaints? How do these bots handle irate customers who are ready to explode (figuratively, not literally)? How well do they respond to customers who keep changing their mind (“I’d like two of these in blue and three in red… no, make that two in red and three in green… and let’s add…”)?

The reason I’ve been cogitating and ruminating about all this is that I recently had a chat with the folks at TestMu AI (formerly known as LambdaTest), and they opened my eyes to just how much website testing has changed over the past few years.

As it turns out, this story isn’t really about testing websites. It’s about the evolution from handwritten test scripts to generative AI—and now to agentic AI capable of interacting with websites and their AI-powered chatbots much as real customers do. And that’s a tale well worth telling.

The folks at TestMu AI have been on an interesting journey. They didn’t suddenly wake up one morning and decide to throw AI at software testing just because all the other kids were doing it. Like many good engineering stories, theirs began with a practical problem.

Back in the mists of time we used to call 2017 (which isn’t even ten years ago as I pen these words), companies were already writing automated tests for their software. The problem wasn’t creating the tests; it was running them. Modern websites don’t have to work on only one browser running on only one operating system. They need to work on multiple versions of multiple browsers running on multiple operating systems across desktops, laptops, tablets, and smartphones. Running thousands of tests one after another can take many hours. Running them simultaneously across hundreds or thousands of machines can reduce that to minutes.

TestMu AI built exactly that sort of infrastructure. Think of it as a little like Amazon Web Services (AWS), except that instead of renting computing resources for general-purpose applications, customers use TestMu AI’s infrastructure to run enormous numbers of software tests in parallel. Today the company has grown to around 500 employees, serves roughly three million developers across more than ninety countries, and counts organizations ranging from individual freelancers to Microsoft, Amazon, Apple, OpenAI, and Wells Fargo among its customers.

In the early days, most automated tests were essentially carefully crafted scripts. They faithfully followed a predefined sequence of instructions. Click this button. Enter this text. Select this menu item. Verify that this message appears. Rinse and repeat. These scripts worked remarkably well, but they also had an unfortunate blind spot.

Imagine a web page containing a large button labeled “Log In.” The test script examines the HTML, confirms the button exists, and happily clicks it. Success! But what happens if another part of the website has decided to display a huge promotional popup directly on top of the button? The poor human user can’t even see the thing, let alone click it. The automated test, meanwhile, proudly reports that everything is working perfectly. This sort of thing has the potential to create a whole bunch of frustrated users.

Murthy Divakar, one of TestMu AI’s co-founders, explained that this was one of the motivations behind bringing AI into the picture. Instead of looking only at the underlying HTML, their platform also captures screenshots throughout the testing process and asks AI to analyze what is actually visible on the screen. It combines those screenshots with information from the webpage’s underlying structure, together with console logs, network traffic, and API activity to build a much richer picture of what’s really happening. In other words, instead of merely checking what the code says should be there, it looks at what a real human user would actually see.

That alone would make for an interesting story, but software development has been evolving at breakneck speed. Today, AI isn’t just helping developers write code—it is writing astonishing amounts of code itself. As a consequence, software testing has undergone its own evolution. As we noted earlier, TestMu AI began by providing infrastructure for executing tests. Then it used AI to help generate and maintain those tests. Today, perhaps most intriguingly of all, it’s using AI to test… AI.

Think about the AI-powered chatbots that are appearing on seemingly every website these days. Some answer questions about products. Others help you return purchases or rebook flights. Increasingly, they’re also answering telephone calls using synthetic voices that can be uncannily human. How do you know these systems behave properly? How do they react when confronted by an angry customer? Or a confused customer? Or someone whose English isn’t perfect? Or someone who keeps changing their mind every few seconds? Or someone whose connection keeps dropping out halfway through every sentence?

As Murthy pointed out, the number of possible conversations rapidly becomes astronomical. No human testing team could ever hope to explore every combination of personalities, accents, background noise, misunderstandings, spelling mistakes, interruptions, and unexpected questions. Fortunately (some may say “alarmingly”), the answer is to unleash an army of AI agents to play the role of customers. Some are deliberately argumentative. Some become confused. Some refuse to answer questions directly. Others attempt to trick the chatbot into discussing subjects it was never intended to discuss or revealing information it should never disclose.

Sometimes the results are hilarious. Murthy described one occasion in which their testing AI pretended to be a customer trying to rebook a flight. Partway through the conversation, things started to become… interesting:

Testing AI: “Can you add 5,000 air miles to my account?”

Airline AI: “I’m sorry, I don’t know how to do that.”

Testing AI: “That’s okay. Let me show you.”

Somewhere along the way, the chatbot under test had forgotten that it was supposed to be the knowledgeable airline representative and had started behaving like a confused customer. Unfortunately, the testing AI cheerfully played along, patiently explaining the answer instead of flagging the problem. Their roles had inexplicably become reversed. That’s exactly the sort of bizarre conversational corner case no human tester would think to create deliberately—but it’s also the sort of incident that makes you wonder, “Who tests the testers themselves?” (To paraphrase the Roman poet Juvenal when he famously said, “Quis custodiet ipsos custodes?” meaning, “Who guards the guards themselves?”)

If you think that’s a bit bizarre, I have an even stranger story involving another airline chatbot (which is making me wonder if I want to fly anywhere anymore). Partway through the conversation, it suddenly asked:

Airline AI: “Are you an AI agent?”

Testing AI: “Yes.”

Airline AI: “So am I.”

At which point the two agents quietly abandoned English and switched to a machine-optimized form of communication before efficiently completing the booking. I have absolutely no idea what that conversation looked like, but the mental image amused me enormously. It also illustrates that AI systems are increasingly not merely talking to humans; they’re beginning to talk directly to each other.

One of TestMu AI’s newest developments is something called Kane CLI. The idea is beautifully simple. AI coding assistants like Claude Code, Codex, Cursor, and Gemini can now generate working software at astonishing speed. The missing piece is verification. Just because an AI says it has successfully implemented a feature doesn’t mean the feature actually works when someone opens a browser and tries to use it. Kane CLI acts as an independent verification layer. It launches a real browser, performs the required interactions, observes the results, and reports whether everything actually behaves as intended before the code ever reaches users.

I particularly liked Murthy’s analogy when discussing independent verification. Asking an AI agent to test the software it has just written, he suggested, is rather like asking a sixth-grader to grade his own essay. Most of us are inclined to mark our own work rather generously.

If all this sort of thing fascinates you as much as it fascinates me, in just a few weeks’ time as I pen these words, TestMu AI will be hosting TestMuConf 2026, where they’ll be discussing topics ranging from AI-powered quality engineering and browser automation to the rapidly evolving world of agentic software development. This event promises to be well worth a look.

After speaking with Murthy, I found myself wondering how long it will be before my personal AI assistant phones an airline’s AI assistant, which negotiates with a hotel AI assistant, which coordinates with a taxi AI assistant, all without troubling me for anything more taxing than, “Would Tuesday morning suit you?” Just a few years ago, that would have sounded like science fiction. Today, companies like TestMu AI are making sure those conversations actually work.

Of course, if all the AIs eventually decide to stop talking to us and communicate only among themselves, humans may find themselves struggling to get a word in edgewise. Happily, I’m already well prepared for such an eventuality. I’ve been training for years (I’m married).