Across one of our agents, the large majority of genuine defects - eight out of nine, when I went back and classified them - belonged to a single category. Not off-by-one, not a race, not a bad regex.

The category is: the code was written correctly, it was tested correctly, and it never executed on the path that mattered.

Every one of them looked healthy on a dashboard. Every one had passing unit tests. The tests passed because they called the function directly, and the function was fine. The wiring was not.

Case 1: the feature that shipped switched off

We publish forecast intervals, and at some point we added conditioning: instead of reading historical quantiles across all market conditions, filter the sample to periods resembling the current one. It matters a lot - unconditional intervals carry a permanent allowance for turbulence that a calm market has not earned.

The code was written. It was correct. It had a toggle:

useConditioning = input.bool(false, "Condition on volatility regime")

Enter fullscreen mode Exit fullscreen mode

That false is the entire bug.

The mechanism existed, its tests passed, and it never ran. Every chart we published for weeks drew the unconditional band while our documentation described the conditioned one. Nothing errored. Nothing looked wrong. Coverage came out at 84% against a stated 50%, and because over-coverage produces no failures - every outcome lands inside a too-wide band - there was no symptom to investigate.

A default value is a code path. Treat it as one.

Case 2: the function that only advances when its branch is taken

This one is language-specific but the shape generalises, and it is nastier because there is no toggle to find.

Some functions carry internal state across invocations. Moving averages, correlations, anything that maintains a rolling window. In Pine Script these are the ta.* family, but the pattern exists anywhere you have a stateful helper that assumes it is called once per tick.

Write this and it looks fine:

ma(src, len, kind) =>
    kind == "EMA" ? ta.ema(src, len) : ta.sma(src, len)

Enter fullscreen mode Exit fullscreen mode

It compiles. It returns plausible numbers. It is wrong.

Only one branch executes per bar, so only one of the two averages advances its internal state. The other one is being fed a history with holes in it. The value it returns is not the moving average of the series - it is the moving average of the subset of bars where that branch happened to be taken.

The fix is to compute both unconditionally and select afterwards:

ma(src, len, kind) =>
    e = ta.ema(src, len)
    s = ta.sma(src, len)
    kind == "EMA" ? e : s

Enter fullscreen mode Exit fullscreen mode

Slightly more arithmetic, correct answers. I found this in two separate places in the same codebase on the same day, which tells you how natural the wrong version feels to write.

Case 3: the check that checked the wrong thing

A smaller one, from the same week, because it completes the pattern.

We had a text sanitiser that converts typographic characters to ASCII, because certain publishing platforms decode UTF-8 as a legacy codepage and an em dash arrives as mojibake. Fine. It also contained this:

while "  -  " in text:
    text = text.replace("  -  ", " - ")

Enter fullscreen mode Exit fullscreen mode

The intent was to clean up the double space left behind when a spaced em dash becomes a spaced hyphen. The effect was to collapse aligned indentation in every file it touched. Run in check mode it reported false positives on source files. Run in fix mode it would have silently mangled them.

The rule was correct for the case it was written for and wrong for every other input. A cleanup step that runs globally is not a cleanup step, it is a transformation.

Why unit tests do not catch any of this

Because unit tests call the function.

A test for the conditioning logic imports the conditioning function, feeds it data, and asserts the output. It passes. It says nothing about whether production reaches that function. A test for ma() calls ma() with kind="EMA" and gets a correct EMA, because in that test every invocation takes the EMA branch and the state advances properly.

The defect lives in the relationship between components, and unit tests are specifically designed not to look there. That is usually a virtue. Here it is the blind spot.

What actually finds them

Three things, in order of how much they have paid off for us.

Review the call graph, not the components. The productive question is not "is this correct" but "under what conditions does this execute, and did I verify it under those conditions". For every function you care about, trace backwards to the entry point and check that the path is reachable with production configuration.

Treat every default as a decision. Any flag, any optional parameter, any if enabled branch. Write down what runs when nobody touches anything, because that is what runs.

Assert on observable output, not on internals. Our conditioning bug would have been caught in a day by a check that compared the published band width against the conditioned band width and complained when they matched. That check is trivial and we did not have it, because we knew the code was correct.

The cheap version

If you take one thing: after adding a feature behind a flag, grep for the flag and read every line that references it. Not to check the logic - to check that production sets it.

That is a two-minute habit and it would have saved us several weeks of publishing numbers that quietly described a different calculation than the one we were documenting.

The bug was never in the code. It was in the assumption that written means running.