Stop Stuffing Your LLM Agent's Context Window: Structured Memory Categories with Mem0

Most tutorials on giving an LLM agent "memory" show you the same three lines:

m = Memory()
m.add("User likes dark mode", user_id="alice")
m.search("What does the user prefer?", user_id="alice")

Enter fullscreen mode Exit fullscreen mode

This works in a demo. It falls apart in a real agent that runs for weeks, because it treats every fact as equally important and equally permanent. In practice, an agent accumulates at least four different kinds of memory that decay, get retrieved, and get invalidated in completely different ways. If you store them all the same way, you get one of two failure modes: the agent re-reads stale project state as if it were still true, or it drowns its context window in low-value trivia every time it calls search().

This article walks through a typed memory schema on top of Mem0 that fixes both problems, with working code.

Why "just store everything" breaks down

Say your agent is a coding assistant working across sessions on the same repo. Over a few weeks it will learn things like:

  • "The user is a backend engineer, new to React."
  • "Don't mock the database in integration tests — we got burned last quarter."
  • "The auth rewrite is blocked on a legal review, ETA next Thursday."
  • "Bug reports are tracked in the INGEST project in Linear."

These look similar as text, but they behave completely differently:

  1. User facts are stable — they rarely change and should almost always be retrieved.
  2. Feedback/corrections are behavioral rules — they need to be applied silently, not surfaced as trivia.
  3. Project state decays fast — "ETA next Thursday" is false a week later and actively harmful if retrieved after it's stale.
  4. References are pointers to external systems — useless without the context of when they're relevant.

A flat memory.add(text) call has no way to express this. When you later call search(), Mem0's relevance ranking will happily surface a three-week-old "ETA next Thursday" note alongside a permanent user preference, because both score similarly on semantic similarity to your query.

A typed schema using Mem0's metadata

Mem0's add() accepts arbitrary metadata, and search()/get_all() support filtering on it. That's enough to build a lightweight type system without touching Mem0's internals.

from mem0 import Memory

m = Memory()

def remember(text, user_id, kind, **extra):
    """kind: 'user' | 'feedback' | 'project' | 'reference'"""
    m.add(
        [{"role": "user", "content": text}],
        user_id=user_id,
        metadata={"kind": kind, **extra},
    )

remember(
    "User is a backend engineer, new to the React side of this repo.",
    user_id="alice",
    kind="user",
)

remember(
    "Don't mock the database in integration tests — a mocked/prod "
    "divergence masked a broken migration last quarter.",
    user_id="alice",
    kind="feedback",
    scope="testing",
)

remember(
    "Auth middleware rewrite is blocked on legal review of session "
    "token storage. Target: 2026-08-06.",
    user_id="alice",
    kind="project",
    expires="2026-08-13",
)

remember(
    "Bug reports live in Linear project 'INGEST'.",
    user_id="alice",
    kind="reference",
)

Enter fullscreen mode Exit fullscreen mode

Retrieval now becomes a two-step process instead of one blind semantic search: pull relevant memories by kind, then let the LLM decide how to use each type.

def load_context(user_id, query):
    facts = m.search(query, user_id=user_id, filters={"kind": "user"})
    rules = m.get_all(user_id=user_id, filters={"kind": "feedback"})
    state = m.search(query, user_id=user_id, filters={"kind": "project"})
    return {
        "facts": [r["memory"] for r in facts["results"]],
        "rules": [r["memory"] for r in rules["results"]],
        "state": [r["memory"] for r in state["results"]],
    }

Enter fullscreen mode Exit fullscreen mode

You now compose the system prompt from three distinct sections instead of one undifferentiated memory dump — "here's who the user is," "here are standing rules you must not violate," "here's what's currently in flight." Feedback-kind memories, in particular, should be injected unconditionally near the top of the system prompt rather than semantically retrieved — a correction like "don't skip pre-commit hooks" needs to apply even when the current query has no lexical overlap with "hooks."

Handling decay: the part most integrations skip

Project-state memories are the ones that cause real bugs when stale. Mem0 doesn't auto-expire memories, so build expiry into your read path, not just your write path:

from datetime import date

def load_project_state(user_id, query):
    results = m.search(query, user_id=user_id, filters={"kind": "project"})
    today = date.today().isoformat()
    fresh = []
    for r in results["results"]:
        expires = r.get("metadata", {}).get("expires")
        if expires and expires < today:
            m.delete(r["id"])   # prune, don't just skip
            continue
        fresh.append(r["memory"])
    return fresh

Enter fullscreen mode Exit fullscreen mode

Pruning on read (rather than a separate cron job) keeps the store self-cleaning without extra infrastructure, and it means your token budget for the "project state" section of the prompt never grows unbounded.

The retrieval mistake that costs the most tokens

The single biggest inefficiency I've seen in Mem0 integrations is calling search() once per turn with the raw user message as the query, at top_k defaults, for every memory kind. That's 3-4 vector searches and several KB of retrieved text per turn, most of which is irrelevant to a short follow-up question like "did that work?"

Two cheap fixes:

  • Cache feedback-kind memories in-process for the session instead of re-querying Mem0 every turn — they change rarely and should be loaded once at session start.
  • Skip the project-kind search entirely on turns where the query doesn't reference ongoing work (a simple keyword gate — "status," "still," "blocked," "when" — is enough to cut a third of your searches without hurting recall).

Results

Splitting memory by kind instead of using one flat store did three concrete things in a long-running coding agent I maintain: it stopped stale "in progress" notes from being read back as current fact, it let behavioral corrections apply consistently instead of depending on semantic luck, and it cut average per-turn retrieved-memory tokens by roughly 40% by making expiry and caching possible in the first place. None of this requires anything beyond what mem0ai already exposes — metadata and filters are enough to build a real type system on top of a memory store that, out of the box, treats every fact the same.

If you're integrating Mem0 into an agent that's meant to run for more than a single session, the schema is the part worth designing deliberately — the SDK calls themselves are the easy part.