An agent-native product isn't a chatbot bolted onto a CRUD app — it's an MCP server that lets an agent do things. Read invoices, create expenses, mark a client overdue. The demo is easy. The part that decides whether the thing survives contact with real traffic is the layer nobody screenshots: what happens when a write is sent twice.

This is the unglamorous half of agent-native engineering. Every write operation an agent can invoke needs idempotency, safe retries, and typed recoverable errors — not as a nice-to-have, but as the difference between a product and a liability. Here's why, and how to build it.

Agents turn "at-least-once" from a footnote into a daily event

Distributed systems people have known forever that networks give you at-least-once delivery, not exactly-once. A request goes out, the response gets lost on the way back, the caller doesn't know if the write landed, so it retries. With a human clicking a button, this is rare enough to hand-wave.

Agents remove the hand-wave. Three things about how agents call tools push duplicate writes from "edge case" to "Tuesday":

  1. They retry automatically. A well-behaved MCP client backs off and retries on 429 and 5xx. The Frihet MCP server (github.com/Frihet-io/frihet-mcp) enforces 100 requests per minute per fri_ key, and its own client retries with exponential backoff when it hits that ceiling. A retry after a 429 is supposed to happen — which means every write behind that endpoint has to survive being delivered more than once.
  2. They time out and resume. An agent loop hits a wall-clock limit mid-tool-call, the session gets resumed, and the planner — having never seen a result — reasonably decides to "create the invoice" again.
  3. They fan out. Give a model a goal ("bill everyone for last month") and it will happily issue parallel tool calls. Two of them racing on the same logical write is now a concurrency problem, not a theoretical one.

MCP itself does not save you here. A tool call is just RPC over JSON — the protocol says nothing about whether invoking create_invoice twice creates one invoice or two. That contract is entirely yours to define on the server. If you don't define it, you've defined it as "two."

For a read tool, a duplicate is free. For a write tool in a fiscal domain — an ERP that prepares VeriFactu records, IVA and IRPF filings — a duplicate is a phantom invoice in someone's tax quarter. The blast radius is exactly why the boring layer matters.

Idempotency keys: the client names the request, once

The fix is to let the caller assign a stable identity to a logical operation, independent of how many times it's physically delivered. That's an idempotency key: a client-generated token (usually a UUID) that stays the same across retries of the same intended write, and changes for a genuinely new one.

The rule for the client is simple: generate the key once, at the point of intent, and reuse it for every retry of that same intent. If the agent decides to bill a client, it mints one key and carries it through all four delivery attempts. When it decides to bill a different client, it mints a new one.

The server's job is to make the key mean something: the first time it sees a key, do the work and remember the result; every subsequent time, return the remembered result without touching the database again.

The subtlety that trips people up is atomicity. "Check if the key exists, then insert it" is a race — two concurrent retries both read "not found" and both do the work. The claim has to be a single atomic operation, which in Postgres is an INSERT ... ON CONFLICT DO NOTHING and in most KV stores a compare-and-set.

import { createHash } from "node:crypto";

interface WriteContext {
  accountId: string;      // scope keys per tenant — never globally
  toolName: string;       // e.g. "create_invoice"
  idempotencyKey: string; // client-supplied, stable across retries
}

async function runOnce<T>(
  ctx: WriteContext,
  payload: unknown,
  work: () => Promise<T>,
): Promise<T> {
  const slot = `${ctx.accountId}:${ctx.toolName}:${ctx.idempotencyKey}`;
  const fingerprint = createHash("sha256")
    .update(JSON.stringify(payload))
    .digest("hex");

  // Atomic claim: only the FIRST caller inserts the row.
  const claim = await store.claim(slot, fingerprint); // INSERT ... ON CONFLICT DO NOTHING
  if (!claim.inserted) {
    // Key already seen. Same body  -> return the stored result.
    // Different body -> the client reused a key for a new request. Reject.
    if (claim.fingerprint !== fingerprint) {
      throw new ToolError("idempotency_key_reused", {
        retryable: false,
        message: "Idempotency key was reused with a different payload.",
      });
    }
    return store.awaitResult<T>(slot); // return the SAME result, never re-run
  }

  try {
    const result = await work();
    await store.complete(slot, result);
    return result;
  } catch (err) {
    await store.fail(slot); // release the slot so a genuine retry can proceed
    throw err;
  }
}

Enter fullscreen mode Exit fullscreen mode

Three things this snippet gets right that naive versions miss:

  • The key is scoped per tenant. Two accounts can generate the same UUID; without accountId in the slot, one customer's retry could return another's invoice. Scope aggressively.
  • The payload is fingerprinted. Reusing a key with a different body is almost always a client bug. Returning the old result silently would hide it; creating a new record would defeat the point. Rejecting it with a typed error is the honest answer.
  • The stored result is returned verbatim. The whole contract is that the second caller gets exactly what the first caller got — same invoice id, same numbers — not a fresh write that happens to look similar.

How long do you keep the record? Long enough to outlive any retry storm — hours to a day is typical, and it's a deliberate trade-off, not a default. Too short and a slow retry slips through the window and double-writes; too long and you're storing every request body forever.

Safe retries need a taxonomy, not a try/catch

Idempotency makes retrying safe. It doesn't tell you when to retry. That decision depends entirely on what went wrong, and "an error happened" is not enough information to make it.

  • A 429 or 503 is transient — back off and retry.
  • A 422 validation_failed is permanent — retrying the same bad payload a thousand times just burns your rate limit against a wall.
  • A 504 gateway timeout is the genuinely hard one: the write may or may not have landed. This is the exact case idempotency was built for. You retry with the same key, and the server either finishes the original work or hands back the result it already produced. Without the key, a retry here is how you double-bill someone.

For the agent to make this call, the error has to be machine-readable. A stringified 500 Internal Server Error forces the model to guess from prose, and models guess wrong. Ship a typed error taxonomy instead: a stable code, an explicit retryable boolean, and a hint for how long to wait.

type ErrorCode =
  | "rate_limited"          // 429 — back off, then retry
  | "upstream_timeout"      // 504 — retry with the SAME idempotency key
  | "validation_failed"     // 422 — do NOT retry; the payload is wrong
  | "insufficient_funds"    // 402 — do NOT retry; surface to the user
  | "idempotency_key_reused"//     — client bug; do NOT retry
  | "conflict";             // 409 — a concurrent write won; re-read, then decide

class ToolError extends Error {
  constructor(
    public readonly code: ErrorCode,
    public readonly meta: {
      retryable: boolean;
      message: string;
      retryAfterMs?: number;
    },
  ) {
    super(meta.message);
  }
}

// The client/agent reads `retryable` — it never parses a stringified 500.
function isRetryable(err: unknown): err is ToolError {
  return err instanceof ToolError && err.meta.retryable;
}

Enter fullscreen mode Exit fullscreen mode

Pair the taxonomy with exponential backoff and jitter. Backoff without jitter means every one of your fanned-out tool calls that got rate-limited retries at the same instant — a synchronized stampede that trips the limit again. Randomizing the delay spreads the herd out. And cap the attempts: retryable does not mean retry forever.

Note that retryable lives on the server's side of the contract but is consumed by the client. That's the point — the server knows whether a write is safe to repeat, so it says so explicitly instead of making every caller reverse-engineer the answer from an HTTP status and a hope.

Why this is the whole game for agent-native products

A demo proves an agent can create an invoice. Production asks a harder question: when the same instruction gets delivered twice — because a network blipped, a session resumed, or a 429 triggered the retry the client was designed to perform — does your customer end up with one invoice or two?

Consider the surface area. A server like Frihet exposes 157 tools over mcp.frihet.io, a large share of them writes into an ERP that generates real tax records. Every one of those write tools is a place where at-least-once delivery meets a fiscal side effect. Idempotency keys, atomic server-side dedup, and typed recoverable errors are what turn that surface from a duplicate-data generator into something an agent can hammer safely at 100 requests a minute. That contract is the argument of this article: it is what every server on the other end of an agent owes its callers. It is also the least glamorous work in the codebase and the work that determines whether the product is trustworthy.

The industry spent a decade learning this for payments — it's why Stripe made idempotency keys a first-class header. Agents make the lesson urgent everywhere at once, because agents retry, resume, and fan out by default. If you're building anything an agent can write to, treat this as the foundation, not the polish. The chatbot is the part users see. This is the part that keeps their data correct.

Next up: designing tool schemas an agent won't misuse — types, enums, and error contracts as the real interface.