Shiv Shankar

LLMs Are Stateless. Here Is What That Actually Means for Agent Memory.

There is a sentence that most AI agent tutorials skip over, and it explains
why building reliable long-running agents is so hard:

Between API calls, the model knows nothing.

Every "memory feature" you have ever seen — ChatGPT's memory, Claude's Projects,
LangMem, Mem0 — is a human-written system that prepares text and injects it into
the context window before calling the model. The model itself has no persistent state.

The implication people miss

If the LLM is stateless, then the intelligence of a long-running agent lives entirely
in the harness that manages what gets injected and what gets discarded.

This is not a limitation. It is a design principle.

The research community has started treating it explicitly this way:

  • MemGPT (2023) introduced the "LLM as OS" analogy: the model is a CPU,
    external memory is disk, and the harness manages what pages are in RAM at query time.

  • StateFlow (2024) goes further: the agent is a Finite State Machine.
    The LLM is only invoked at state transitions, not held responsible for state itself.

  • ClawVM (2026) formalises this as a virtual memory layer: the harness enforces
    deterministic, validated writebacks at every lifecycle boundary.
    The LLM outputs raw thoughts; the harness decides what is committed.

The consequence for memory design

If the LLM is purely a processing unit, then the quality of your agent's memory
is entirely the quality of your harness — the system that decides what facts to
store, how long they are valid, and when they should be superseded.

A harness that stores flat embeddings and retrieves by similarity gives you a
mediocre agent. A harness that stores temporal assertions and resolves conflicts
deterministically gives you a reliable one.

What "temporal assertion" means in practice

A temporal assertion is a fact with a lifespan:

subject:   user_project
predicate: uses_language  
object:    Python
valid_from: 2025-01-10
valid_to:   2025-06-15   ← closed when user switched to TypeScript

Enter fullscreen mode Exit fullscreen mode

The harness stores this, not the LLM. The LLM never sees the old fact again
once valid_to is set. Active state is WHERE valid_to IS NULL.

This is the model behind Smriti — an open-source
temporal memory engine built precisely to be a reliable harness for stateless LLMs.

The LLM is not the agent. The harness is the agent. Build the harness well.

MIT license. PostgreSQL-backed. Open source.