Venkatesh S

I spent ten years building enterprise Java systems — a payments monolith broken into Spring Cloud microservices, banking applications, and currently a Cisco security platform. More recently I moved into building LLM systems seriously. The surprise wasn't how much I had to learn. It was how much transferred.

Agentic systems are distributed systems. The vocabulary is new; the failure modes are not.

Timeouts and retries — same problem, worse defaults

A Java service that calls a flaky dependency without timeouts is a junior-engineer mistake. An agent that calls an LLM without them is somehow normal. LLM calls are slow, occasionally hang, and fail in novel ways (malformed JSON, refusals, half-finished tool calls). Everything Resilience4j taught me applies directly: bounded retries with backoff, circuit breakers that fail fast with an honest error instead of silently substituting something expensive, and — this one is new — semantic failure detection, because an LLM can return HTTP 200 with garbage inside.

In Atlas, the gateway circuit-breaks to 503 + Retry-After when the model endpoint is down. That is deliberately boring. Boring is the point.

Idempotency — now with side effects an LLM chose

Backend rule: any handler that can be retried must be idempotent. Agent rule: same, except the retry might be the planner deciding to call the tool again because the first response "didn't look right." If your agent's tools mutate state, you need idempotency keys and an audit trail more than a chat UI needs streaming.

Atlas routes every write through an MCP tool server with audience-scoped tokens (RFC 8707) and an append-only, hash-chained audit log, behind a human-in-the-loop checkpoint for governed actions. That design is not AI innovation — it's what any bank-grade action API looks like. The innovation is refusing to drop the standard just because an LLM is the caller.

Cost budgets are the new capacity planning

In backend work you plan capacity: connection pools, thread pools, rate limits. In LLM systems the scarce resource is spend. The same machinery applies — token-bucket rate limiting, a per-user daily budget guard (in Atlas: Redis-backed, keyed per user per UTC day), alerting at a spend ceiling — plus one new lever: routing. Atlas routes simple queries to a small model and reserves larger models for multi-step reasoning, the way you'd route reads to a replica. Cost regression is gated in CI like a latency budget.

State machines beat vibes

LangGraph clicked for me precisely because it isn't magic: it's an explicit state graph with checkpoints — nodes, edges, persisted state, resumability. If you've designed a workflow engine or a saga with compensation steps, you already think this way. The planner–executor pattern with a durable human-approval checkpoint is a state machine an enterprise architect would recognize from a 2010 BPM diagram. That's a compliment.

What's genuinely new

Honesty requires the other list. Three things had no backend equivalent and took real work:

  1. Evaluation is statistical, not binary. Tests pass or fail; eval metrics move in distributions. Learning to set floors and regression bands (and when to reach for paired bootstrap and McNemar) was new muscle.
  2. The retrieval layer is a ranking problem. Hybrid search, reranking, and abstention thresholds are closer to information retrieval than to anything in Spring.
  3. Prompt injection inverts the trust model. Backend security trusts your own data; RAG security cannot — retrieved documents are untrusted input to the model. Spotlighting and quarantine have no REST-era analogue.

The pitch, condensed

If you're a backend engineer eyeing AI engineering: your instincts about timeouts, idempotency, budgets, audit, and state are not legacy baggage — they're the exact discipline most LLM systems are missing. Learn retrieval, learn evals, keep the discipline.