Ted

I went looking for one wrong email address and found six wrong domains.

They had been in production for months, across six different server functions. Nothing had ever complained, because there was nothing in the stack capable of complaining. Every one of them was valid code.

What was actually there

Three distinct hosts, none of which I own.

The first was the .com version of my own brand, which I have never owned — the site runs on a different TLD. Somewhere along the way, generated code decided the .com was the obvious address and used it anyway: in a contact-form reply telling users to email support there, and in admin links inside security alerts.

The second was the .io version. Same idea, different guess. This one was worse, because it was not only a link:

from: "Alerts <[email protected]>",

Enter fullscreen mode Exit fullscreen mode

That is the sender on a batch of subscriber notifications. Transactional email providers verify sending domains. You cannot send from a domain you have not proven you control. So every one of those emails failed at the provider, silently, from the day the code shipped.

The third was the provider's own shared test domain — the address used in every tutorial and quickstart:

from: "App <[email protected]>",

Enter fullscreen mode Exit fullscreen mode

That one was on the password reset email. Shared test domains are heavily restricted, typically to your own account address. So the one email in the system that a locked-out user depends on was going out from the least deliverable sender available.

Why none of it was caught

Consider what each stage of a normal pipeline actually knows.

The typechecker knows "[email protected]" is a string, and a string is what the field wants. Correct, and finished.

The bundler knows the file parses. Correct, and finished.

The tests know the function returns the shape they asserted. If they mock the mail client — and they do, because nobody sends real email in CI — they cannot see the sender at all.

The deploy knows the function compiled and uploaded.

Not one of these stages holds a fact about which domains I own. That is not an oversight in any of them. Type systems check the shape of values, not their correspondence to the world. There is no type that means "a domain this organisation controls."

So the whole apparatus is checking grammar. The sentence is well-formed. Nobody is checking whether it is true.

Generated code fails where the plausible answer is wrong

This is the part that has changed for me.

When I write a domain by hand, I am reading it off something — a dashboard, a DNS record, an invoice. When a model writes one, it is producing the most plausible token sequence given the context. For a brand named X, x.com is overwhelmingly the most plausible domain. For an email provider's sender field, the address from the documentation is the most plausible value.

Both guesses are excellent guesses. They are what a reasonable person would assume. They are also wrong, and wrong in a way that is invisible in review, because [email protected] looks correct on the page unless you already know the answer.

That predicts where to look. These errors cluster wherever the correct value is guessable: brand domains, support addresses, affiliate and partner IDs, API senders, dashboard URLs. Anywhere there is an obvious-looking answer, the obvious-looking answer is what you get, and it is right often enough that the exceptions never get checked.

It is not a hallucination in the dramatic sense. Nothing is invented. Something real is substituted for something true.

Declare your external surface

The fix took about twenty minutes. Extract every URL host and email domain from the source, compare against a list of hosts you have declared, fail on anything else.

✗ 12 undeclared domain reference(s) in git ref abc1234:

  resend.dev  (5 references)
      functions/send-password-reset/index.ts:184
      functions/verify-code/index.ts:155
      … and 3 more

  brand.io   (3 references)
  brand.com  (3 references)

Enter fullscreen mode Exit fullscreen mode

Run against the commit from before I started fixing things by hand, it reported all eleven references I had found — plus one more I had missed entirely. A page was setting its canonical URL to a domain I do not own, which meant it had been telling search engines that the authoritative copy of that page lives on someone else's property.

I found six by reading. The script found seven, in about a second, and will keep finding them.

Two details matter more than the regex:

The allowlist belongs in the repository, not in configuration. The failure mode is generated code introducing a plausible host. If the allowlist lives somewhere an assistant can quietly extend, it stops being a check and becomes a formality. In the repo, widening it is a diff, and a diff gets looked at.

Where it runs is not obvious. A pre-push hook is the tight loop, and it only sees commits that pass through your machine. If you use a hosted agent that commits and pushes on its own — mine does — a local hook is blind precisely where most of the generated code enters. So the same script also runs on a schedule against the remote branch, and alerts only when the set of offending hosts changes, so a known-unfixed finding does not decay into noise.

What it does not do

It catches wrong constants. It does not catch wrong behaviour.

The same week, I fixed a lookup that had been failing on every call for months because it queried a schema the API layer does not expose. Valid code, real function, correct arguments, and the error was being discarded so the failure read as an ordinary result. No domain check would ever see that. It is a different class and it needs a different tool.

I would rather say that plainly than let a script imply more coverage than it has. A guard that quietly under-delivers is the thing we are trying to get away from.

The general version

There is a category of error that generated code produces far more readily than handwritten code, and that conventional tooling is structurally unable to detect: values that are well-formed and untrue.

You cannot review your way out of it, because the values look right. You cannot typecheck your way out of it, because they are the right type. What you can do is write down the small set of facts your project depends on — which domains are yours, which accounts, which endpoints — and make a machine compare the code against that list every time.

The list is the interesting artifact. Not because it is clever, but because until you sit down to write it, nobody has ever actually enumerated what your codebase is allowed to talk to.