Venkatesh S

Most RAG demos have a security model of "none." Documents go into one shared index; anyone who can ask a question can surface content from any document. In a compliance or financial domain, that's not a rough edge — it's a disqualifier.

When I built Atlas, an enterprise copilot for a compliance domain, the first architectural decision was where access control lives. There are three options, and two of them are wrong.

Option 1 (wrong): filter after generation

Generate the answer, then check whether the user was allowed to see the sources. By then the LLM has already read the restricted content and may have leaked it through paraphrase, summary, or even its refusal ("I can't tell you about the Q3 restructuring memo…" is itself a leak). Post-hoc filtering treats a security boundary like a UX problem.

Option 2 (wrong): filter the final chunk list

Retrieve top-k from the shared index, then drop chunks the user can't access. Better, but it corrupts retrieval quality: your top-k gets silently thinned, and ranking scores were computed against a corpus the user shouldn't see. Worse, timing and scoring side-channels can reveal that restricted documents exist.

Option 3: make clearance part of the retrieval predicate

In Atlas, every chunk in pgvector carries a clearance label (public | analyst | compliance | restricted) as metadata, written at ingestion. Retrieval filters by the caller's verified clearance before ranking. This is the actual shape of the dense retrieval query (sparse tsvector search shares the same predicate builder):

SELECT id, document_id, content, clearance, metadata,
       (1 - (embedding <=> ?::vector)) AS score
FROM atlas_chunk
WHERE clearance = ANY(?)        -- caller's visible labels, enforced BEFORE ranking
ORDER BY embedding <=> ?::vector
LIMIT ?;

Enter fullscreen mode Exit fullscreen mode

One detail worth copying: both retrieval paths — dense kNN and sparse full-text — build their WHERE clause from a single shared RBAC filter builder, so there is exactly one trust boundary and it cannot be bypassed. The predicate is pushed into SQL, never applied as an optional post-filter.

The property this buys you: cross-clearance leakage becomes structurally impossible, not "tested for." The restricted document isn't ranked lower or filtered out — from the query's perspective, it does not exist.

The part everyone skips: proving it stays true

A design is worth little without a regression gate. Atlas has a negative-access hard gate in CI: an eval suite where low-clearance users ask questions whose only correct sources are restricted documents. The passing behavior is a grounded refusal — and the gate fails the build on any leakage. It runs GPU-free against committed cassettes, so it gates every merge like a unit test.

Two implementation notes that mattered:

  1. Clearance comes from the verified identity, not the request. The gateway asserts an independently-verified internal JWT; the RAG engine resolves clearance from that assertion and ignores any client-supplied clearance header when a valid assertion is present. (An attacker who can pass clearance=SECRET as a parameter owns your corpus.)
  2. The semantic cache is clearance-partitioned. A cached answer computed for a high-clearance user must never be served to a low-clearance one. Cache keys include the clearance tier — a leak path that's easy to miss because caching is "just performance."

Takeaway

Ten years of backend work taught me that authorization belongs in the data path, not bolted on after. RAG doesn't change that rule; it just gives you a new data path to forget it in. Put the ACL in the retrieval predicate, then write the eval that fails your build when someone breaks it.