872 million tokens processed — 97.63% of them cached inputs. 5,222 tool calls. 88 issues closed. Zero critical bugs, and several high-severity ones, in a product I had been QA-ing the whole time I built it.

A note before we start. There's nothing exotic in this setup — no Hermes harness, no custom framework. Just Codex and a long prompt. I'm sure a properly configured Hermes run would measure all of this far more rigorously. I just don't think many people have gotten there yet, and I wanted to show what's already reachable without it.

This didn't start as an experiment. It started as nerves.

I was getting ready to run paid ads. Once you start buying traffic, every broken screen costs money — you're paying to deliver strangers to your worst bug. And I had a specific fear: my product runs in multiple languages, and I am not equally fluent in all of them. If something quietly breaks in one locale, I might not notice for weeks. The ads wouldn't stop running. They'd just keep working perfectly, delivering people to something broken.

So instead of clicking around for an afternoon and calling it good, I handed the whole thing to Codex and told it not to stop until it ran out of things to break.

It ran for just under 30 hours.

The instructions

The prompt mattered more than I expected, so here it is, cleaned up. Most of the specificity in it exists because vague instructions produce vague testing.

Objective I'm about to start paid advertising. Before I pay to send strangers to this product, find what they'll hit.

Ground rules

Depth — the happy path is the starting line, not the finish: repeat and double-click and fire concurrent requests; interrupt mid-flow with a refresh, a navigation, a sign-out, a return; switch language mid-session and check that state survives; kill a step deliberately and verify both recovery and usage accounting; mobile, tablet, desktop; keyboard-only, focus, labels, contrast; and check privacy boundaries — can any account see anything it shouldn't?

For every issue Record: what a user would see, exact reproduction steps, root cause in the code, severity (Critical / High / Medium / Low), the fix, the regression test that now covers it, verification after deploy.

Process: Read docs/ first. Fix as you go in small, traceable commits. Deploy to main and re-verify in the live environment. A fix isn't done until it has been checked in production.

Output: A separate report per language, plus one consolidated severity-ranked summary.

Rule 4 and rule 5 are there because of what happened during the run. I'll get to that.

The loop

Deliberately boring:

Observe, Reproduce, Investigate, Fix, Test, Verify, Repeat.

The rule that made the difference was refusing to trust a single pass. A screen working once proves almost nothing. So every flow got revisited after refreshes, sign-outs, restarts, language switches, repeated requests, deliberate failures, and finally, after deployment.

The numbers

Metric Result
Total elapsed time 29h 45m 49s
Recorded agent work time 24h 57m 59s
Major work turns 9
Total tool calls 5,222
Browser-control calls 2,408
Shell / Git / test / deploy calls 2,570
Monitoring and wait calls 134
Images inspected 50
PDF analyses 8
Issues recorded and addressed 88
Total commits 126
Targeted fix commits 111
Automated web checks passed 829
Automated hook checks passed 95

Type checking, linting, builds, and the final test suites all passed before I called it finished. And "finished" didn't mean the last commit, it meant deployed, re-checked in the live environment, and synced back to main.

Yes, it processed 872 million tokens

Here's the headline number, and here's the asterisk, right next to it where it belongs.

Token type Count
Total processed tokens 872,027,184
Input tokens 870,333,094
Cached input tokens 849,674,240
Non-cached input tokens 20,658,854
Output tokens 1,694,090
Reasoning tokens 512,615
Non-cached input + output 22,352,944

872 million is a genuinely fun number to say out loud. It is also 97.63% cached input.

That's what a long-running session looks like from the inside: the same codebase, the same reports, the same accumulated conversation, re-processed on every turn. The cumulative counter faithfully counts all of it. It does not mean 872 million new tokens were introduced or billed as fresh usage.

If you want the number that describes actual new context and generated work, it's roughly 22.35 million, non-cached input plus output.

I'm publishing both, because a large AI usage figure only means something when the caching structure sits beside it. Total volume without the ratio isn't measurement, it's marketing.

Where I had to step in

Two days is long enough that I checked in on it periodically, not reviewing every commit, just watching screens go by and asking myself whether what I was seeing looked right.

The first catch was mine. In every language but one, a particular record displayed correctly. In that one language, it didn't. The agent had looked at that screen, registered it as acceptable, and moved on. I flagged it manually. Once pointed at directly, it was diagnosed and fixed quickly. The fix was never the hard part. Noticing was.

That's the exact failure mode you should expect from an agent working alone: it is very good at "does this throw an error," and much weaker at "does this look wrong compared to the eleven other times I saw something similar." Rule 4 in the prompt above exists because of this.

The second catch came after the run was already declared complete. Going back through the results myself, I found a PostHog event that wasn't firing correctly, the one that records a completed payment.

Think about what that means in context. I was doing all of this to prepare for an ad campaign. The number I would have used to judge whether the ads were working was broken. No screen looked wrong. No test failed. Nothing threw. It would simply have been quietly incorrect forever, on the single metric that mattered most for the thing I was about to spend money on.

That's rule 5. Verify past the screen. A green checkmark that didn't emit its event is not a pass.

What actually changed

88 tracked issues, 111 targeted fix commits. The mismatch is the interesting part: one observed problem usually needed several independent changes. A single inconsistency could touch persisted data, restored UI state, translated output, failure recovery, and test coverage. Rather than folding that into a few fat commits, everything was split into small, traceable units.

The fixes clustered into a few themes:

  • More reliable state restoration when returning to a workflow
  • More consistent handling of repeated or concurrent actions
  • Better recovery when an operation dies partway through
  • Stronger file and document boundary handling
  • Fewer layout breaks across languages and screen sizes
  • Clearer protection against unintended internal-data exposure
  • New automated tests on paths that previously had none

The volume of changed code isn't the point. The point is that every fix traces back to an observed behavior, a reproducible case, and a verification step.

The part that actually bothered me

No critical bugs. Good.

But there were high-severity ones. Plural. In a product where I had been running QA continuously while building it.

That's the finding I keep coming back to, and it's more useful than any of the numbers above. I hadn't been careless. I'd been checking as I went. And a hard 30-hour pass still surfaced multiple high-severity problems that ordinary development QA had walked straight past.

Then the stress campaign itself finished, and I found two more things by looking with my own eyes.

There's a popular idea right now that AI just handles this, that you point a good model at a codebase and quality is a solved problem. This run is strong evidence for how far agents have come, and it's equally strong evidence against that idea. Development-time QA missed things. A 29-hour adversarial campaign caught most of them and still missed the ones a human caught by paying attention. So the honest conclusion isn't "now it's clean." The honest conclusion is:

There are almost certainly still latent bugs in there.

I'll run this again. Right now I believe there is nothing high-severity left. Believe, not know. That distinction is the entire point of writing this up.

What I'd tell someone doing this

Continuity is the capability, not intelligence. A one-off prompt fixes a visible bug. An agent that stays with the product discovers how that bug behaves once you add retries, localization, generated files, and a user who wanders back three steps later.

Browser access and code access are far stronger together. Browser automation tells you something failed. Source access tells you what might fail. Only both together let it see the symptom, trace the implementation, apply the fix, and walk the same journey again to confirm.

A fix without a regression test isn't finished. The screen looking better is not evidence. Where practical, every corrected behavior got a test so a future change can't quietly undo it.

Forbid "probably fine" explicitly. Agents skip small anomalies for the same reason tired humans do, nothing was on fire. Write the rule into the prompt: an inconsistency across languages is a finding, not a mood.

Verify beyond the UI, especially analytics. Events, emails, generated files, and stored state are where silent failures live. These are exactly the bugs that survive a successful test run.

Stay in the loop. The agent extended my reach enormously. It did not take over ownership. Both times something important slipped through, the thing that caught it was a person looking at the screen and thinking hmm, that's odd.

The question worth changing

The most useful outcome here wasn't any single fix. It was watching an agent hold an entire product-stabilization cycle, navigating, inspecting evidence, editing code, running tests, managing changes, waiting on external processes, and coming back to verify, for 30 straight hours.

Which means we've been asking coding agents a slightly small question. Not only:

"Can you build this feature?"

But also:

"Can you keep testing this product, investigate what breaks, fix the root causes, prove the fixes hold, and keep going until the validation criteria are met?"

That's a completely different job description. And right now it may be the most practical one we have, as long as somebody is still watching.

Originally published on Kanapp Notes.