I wired an AI code reviewer into CI, felt clever, and then looked at the bill after a busy week of PRs. Every push, every commit, sending full diffs to a big frontier model. It added up faster than I expected, and most of what it was reviewing was formatting changes and README edits that did not need a genius reading them. So I rebuilt it as two tiers, and the cost dropped hard without losing the catches that mattered.

The idea is simple. Do not send everything to the expensive model. Use a cheap model as a bouncer that decides what is even worth escalating, and reserve the big model (with caching turned on) for the files that could actually hurt you.

Tier one: a cheap triage pass

The first tier looks at the diff and answers one question: is anything here risky enough to justify a real review? That is a classification task, and classification does not need a frontier model.

I run this tier on a small local model. On my own machine that is Ollama with qwen2.5-coder, but in CI it can be a self-hosted small model or the cheapest cloud tier your provider offers. The job is triage, not judgment.

The triage prompt is deliberately narrow. I do not ask it to review. I ask it to route:

You are a triage filter for code review. For the diff below, output JSON only:
{ "risk": "low" | "high", "reason": "<one short phrase>" }

Mark "high" if the diff touches: auth, access control, money/token math,
cryptography, SQL or shell string building, deserialization, file paths,
network calls, or anything that changes who can do what.
Mark "low" for formatting, comments, docs, tests, renames, and config bumps.

DIFF:
<diff here>

Enter fullscreen mode Exit fullscreen mode

A small model is perfectly good at "does this touch auth or money." When it says low, CI posts a one-line "no high-risk changes detected" and stops. No expensive call. In practice that shortcuts the majority of PRs, because most PRs really are docs, tests, and plumbing.

Tier two: the big model, but only on the risky files, with caching

When tier one says high, the risky files get escalated to the strong model. This is where the real review happens, and this is where prompt caching earns its keep.

The trick is that most of what you send the big model does not change between reviews. The system prompt (your review rubric, your severity definitions, your house rules) is identical every time. And your repo context (the surrounding files, the conventions, the interfaces the changed code depends on) is stable across the many small PRs that touch the same area. Caching that stable prefix means you pay full price for it once and a small fraction on every subsequent review that reuses it.

So I structure the request as: cached system prompt, then cached repo context, then the fresh diff last. Anthropic-style caching keys on a stable prefix, so the ordering matters. Stable stuff first, volatile stuff last.

# pseudocode, provider-agnostic shape
response = client.messages.create(
    model="big-model",
    system=[
        {
            "type": "text",
            "text": REVIEW_RUBRIC,            # never changes
            "cache_control": {"type": "ephemeral"},
        },
        {
            "type": "text",
            "text": repo_context_for(files),  # changes rarely
            "cache_control": {"type": "ephemeral"},
        },
    ],
    messages=[
        {"role": "user", "content": build_review_prompt(diff)},  # changes every time
    ],
)

Enter fullscreen mode Exit fullscreen mode

The rubric and the repo context ride the cache. Only the diff is fresh each run. On a repo where lots of small PRs hit the same modules, the cache hit rate is high and the per-review cost of the big model drops to mostly the fresh tokens.

The GitHub Actions sketch

Wiring it together is not exotic. One job, two steps, conditional escalation:

name: ai-review
on: pull_request

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # v4.2.2
        with:
          fetch-depth: 0
          persist-credentials: false

      - name: Triage (cheap model)
        id: triage
        run: |
          git diff origin/${{ github.base_ref }}...HEAD > diff.patch
          python scripts/triage.py diff.patch > triage.json
          echo "risk=$(jq -r .risk triage.json)" >> "$GITHUB_OUTPUT"

      - name: Deep review (big model, cached)
        if: steps.triage.outputs.risk == 'high'
        env:
          MODEL_API_KEY: ${{ secrets.MODEL_API_KEY }}
        run: python scripts/deep_review.py diff.patch

Enter fullscreen mode Exit fullscreen mode

The if: on the second step is the whole cost story. Low-risk PRs never reach the expensive model. High-risk ones get the full treatment.

The catch that diff-only review misses

Here is the part people get wrong, and it is the reason a naive AI reviewer gives you false confidence. If you only send the diff, the model cannot see cross-file bugs.

Say a PR changes a function's return type or the meaning of one of its parameters. The diff looks locally fine. The reviewer, seeing only those lines, approves it. But three other files call that function and now pass the old assumptions, and those files are not in the diff at all. The bug is real, it is severe, and diff-only review is structurally blind to it.

The cheap fix is to include the callers of changed functions in the context you send. You do not need to send the whole repo. You need the changed symbols plus everyone who touches them:

# find who calls the functions changed in this PR
changed_funcs=$(git diff origin/main...HEAD \
  | grep -oP '^\+.*function \K\w+' | sort -u)

for fn in $changed_funcs; do
  grep -rl "$fn(" src/ >> caller_files.txt
done
sort -u caller_files.txt

Enter fullscreen mode Exit fullscreen mode

You feed those caller files into the tier-two context (the part that gets cached, since callers change slowly). Now when a change alters an interface, the model can see the code that relied on the old behavior and flag the mismatch. That single addition catches a class of bug that pure diff review will never see, and because callers are stable, they cost almost nothing after the first cached run.

What this actually buys you

The economics come from routing, not from a cheaper model doing the hard job. The small model is a filter, and filtering is cheap and easy. The big model does the real work, but only on the small slice that deserves it, and it pays full token price only on the changing diff while the rubric and context ride the cache.

I use pieces of this in how spectr-ai reasons about cost too: a fast local pass to decide what is worth a careful, expensive look, then the expensive look only where it counts. Running qwen2.5-coder locally for the triage tier means that stage is effectively free on hardware I already own.

The failure mode to avoid is treating the cheap tier as the reviewer. It is not. It is the bouncer. If your triage model starts writing review comments, you have given a hard job to the wrong tool. Keep tier one narrow, keep tier two well-fed with callers and cached context, and the bill stays small while the catches stay real.

If you run AI review in CI, are you sending bare diffs, or do you pull in the callers of what changed?