A rerun is not automatically a replay

Running the same command twice does not prove that the same execution happened twice.

The contract may have changed. A lockfile may have moved. CI may have selected another backend. A
live API may have returned different state. Even a green result can describe a different operational
world.

Ota 1.6.24 takes a strict position: replay is a comparison against an archived execution witness,
not a synonym for rerun.

The selected workflow, backend, provider, lifecycle, contract snapshot, and captured inputs must be
comparable before Ota will call the result verified.

The command starts from an archived witness

Archive the original execution receipt, then select it as the replay baseline:

ota up --workflow verify --json
ota receipt --json --archive
ota up --workflow verify --replay-baseline latest --json

Enter fullscreen mode Exit fullscreen mode

The replay result is explicit:

  • replay_verified: the selected scope reconciled with the baseline;
  • replay_failed: comparable execution produced material drift or failure;
  • replay_unavailable: Ota could not make an honest comparison.

replay_unavailable is not a weak error message. It is a trust boundary. A baseline from another
workflow, backend, provider, remote target, or lifecycle must not be promoted to last-known-good
evidence for the selected lane.

Declare inputs separately from observed behavior

Ota contracts can name the repo-owned artifacts that define a replay lane:

tasks:
  gate:
    replay_inputs:
      - id: recorded_sql
        kind: static_file
        path: data/fixture.jsonl
      - id: frozen_store
        kind: static_file
        path: data/store.db
      - id: runtime-presentation
        kind: presentation_profile
        path: replay/presentation-profile.yaml
      - id: equivalence
        kind: comparator_profile
        path: replay/comparator-profile.yaml
    witnessed_observations:
      query_traces:
        - id: recorded_queries
          path: evidence/query-trace.jsonl

Enter fullscreen mode Exit fullscreen mode

These are not interchangeable fields.

  • replay_inputs[] records what the selected lane consumed before execution.
  • witnessed_observations.query_traces[] records behavior observed from an execution.

If observed SQL is silently treated as a current-run input, the replay can confirm itself. Ota
keeps the evidence linked but separate so a later comparison can distinguish changed inputs from
changed behavior.

In 1.6.24, Ota captures the observed SHA-256 identity of each declared replay input. That tells a
later replay exactly which bytes the original lane used and exposes identity drift between receipts.
It does not let the 1.6.24 contract declare an expected digest that blocks the very first execution.
That distinction matters: observed identity is execution evidence, not a maintainer-authored pin.

What a verified replay actually says

A verified replay says that Ota reconciled the selected execution scope and the named evidence it
knows how to compare. It does not automatically mean the lane was hermetic.

Matching static inputs remain scoped evidence. Undeclared environment state, live services, remote
APIs, clocks, randomness, and other ambient inputs can still matter. Ota therefore reports replay
posture and hermeticity separately instead of turning one matching hash into a repo-wide claim.

The machine consumer should inspect:

{
  "replay": {
    "posture": "replay_verified",
    "hermeticity": "partly_ambient",
    "baseline": {
      "last_known_good": "replay_verified"
    }
  }
}

Enter fullscreen mode Exit fullscreen mode

The exact payload contains more scope and comparison detail. The important point is that
replay_verified and partly_ambient can both be true. One describes the comparison Ota completed;
the other prevents that result from claiming the rest of the world was frozen.

Bedrock made the distinction concrete

Bedrock, created by Vinicius Pereira, is a natural-language
to SQL stability harness. Its deterministic lane consumes a recorded SQL fixture, a frozen SQLite
store, and a defended baseline. The SQL records also contain useful witnessed query behavior.

That repo forced the product question: should historical model output become a replay input, or
remain witnessed evidence?

Ota keeps both roles explicit. The released-version pressure matrix passed native Ubuntu, macOS,
Windows, and the container replay lane in
run #29572073749. That matrix proves
the bounded offline replay path. It does not prove Bedrock's live model-recording lane or a live
database.

Read the full engineering note: Pressure-testing Ota on Bedrock: query identity as replay
evidence.

The operational rule

Use replay when you need to answer:

Did this selected execution still hold against the archived contract, scope, and named inputs?

Do not use it to imply:

Nothing outside this receipt could have affected the result.

That distinction is why Ota archives execution evidence, preserves semantic contract identity, and
publishes unavailable or partly ambient outcomes instead of manufacturing certainty.

References

Originally posted here: https://ota.run/blog/ota-v1-6-24-replayable-proof-for-real-repository-execution