Permission-Aware Retrieval

Step 1The arithmetic that makes post-filtering indefensible.

Take the obvious design. Embed the query, ask the index for the ten nearest chunks, then drop the ones the user may not read. If that user can see a fraction v of the corpus and permission is roughly independent of relevance, the number of chunks that survive is about k × v.

# post-filter: ask for k, keep what the user may read

visible fraction v    k=10 -> expected survivors
1.0    (admin)                     10.0
0.30   (whole-department reader)    3.0
0.05   (one project)                0.5
0.002  (external contractor)        0.02

# to land n allowed chunks you must over-fetch k ~ n/v

n=10 allowed, v=0.05   ->  k ~    200
n=10 allowed, v=0.002  ->  k ~  5,000

The over-fetch factor you need is 1/v, and v is a property of the user, not of your system. A fixed over-fetch is therefore calibrated for exactly one person in your tenant and wrong for everybody else — generous for the admin who did the testing, catastrophic for the contractor who files the ticket.

It is worse than the independence assumption suggests, because relevance and permission are positively correlated in exactly the cases that matter. The most on-topic chunks for a question about a project usually live in that project's restricted space. So the users with the smallest v are the ones whose best results get filtered out, and the surviving chunks are the generic company-wide boilerplate that was visible to everyone. The answer is not empty. It is confidently vague, which is harder to notice.

Step 2Pre-filtering is the only placement that preserves top-k.

The fix is to make the permission predicate part of the search rather than a stage after it: index the access metadata alongside the vector, and have the engine restrict traversal to candidates that satisfy the filter. Then "top ten" means the ten best chunks this user is allowed to read, which is the thing you actually wanted, and the over-fetch factor disappears because nothing is thrown away after ranking.

What makes this a real engineering decision rather than a flag is that filtered vector search is genuinely hard. An HNSW graph is navigable because its edges were built over the whole corpus; apply a selective filter during traversal and the neighbours you would hop through are excluded, so the walk dead-ends in a region of the graph and returns a short, poor list. Engines answer this differently — maintaining filterable payload indices and falling back to a brute-force scan below a selectivity threshold, or giving each tenant its own physical partition or namespace so that the "filter" is really a choice of index. Which strategy your store uses is one of the axes in choosing a vector database, and it is the axis that this requirement turns from a preference into a constraint.

Before you trust any of it, establish which side of the ranking your engine applies the filter on, because an engine that accepts the filter argument and applies it afterwards is indistinguishable from one that pre-filters until you measure. The test is three lines and worth writing down:

# does your engine pre-filter or post-filter?

1. index 10,000 chunks; tag exactly 20 of them tenant=A
2. query with filter tenant=A, k=10
3. count the results

10 results  -> the filter ran inside the search (pre-filter)
0-2 results -> the filter ran after top-k (post-filter)

# run it again at k=100 on the post-filtering engine and you
# will get a few more. that is the lottery, not a fix.

Step 3The metadata in your index is a cache of the ACL, and revocation is the dangerous direction.

Pre-filtering buys correct ranking, and it buys it by copying authorization state into the retrieval layer. That copy is a cache, so it has a staleness window equal to your sync lag, and the two directions of drift are not symmetric.

• A grant that has not propagated means someone cannot find a document they are entitled to. That is a support ticket. It is annoying and it is visible, because the person knows the document exists and says so.
• A revocation that has not propagated means someone retrieves a document they are no longer entitled to, and the model quotes it into an answer. That is a disclosure, it is silent, and your sync lag is the length of the window.

So treat permission changes as an invalidation event rather than something a nightly job catches up on — the same discipline as index freshness and invalidation, with the difference that here the stale record is a security control rather than a stale paragraph. Then close the window by re-checking the survivors against the authoritative service before they enter the prompt. That final check is affordable precisely because it runs over at most k document ids rather than the corpus: a batch authorization call to whatever owns the truth — a relationship-based service such as OpenFGA or SpiceDB, or your application's own policy layer — costs one round trip and removes the staleness window entirely.

The resulting shape is two enforcement points doing two different jobs, and you need both:

pre-filter   (in the index)  -> correct RANKING
                                cheap, stale, approximate

final check  (vs. authority)  -> correct DISCLOSURE
                                authoritative, fresh, k ids only

# the pre-filter is a performance structure that happens to
# be a permission hint. the final check is the control.
# if the final check drops a chunk, LOG IT -- that is your
# sync lag reporting itself as a number.

That last line is the cheapest observability you will add all quarter. A non-zero rate of chunks dropped by the final check is the only direct measurement of how stale your index's ACL copy really is, and it costs one counter.

Step 4Index groups, not users, and then watch the filter width.

The naive metadata schema stores the list of users who may read each chunk, which makes the index a materialised cross product: a corpus of a million chunks and ten thousand users does not fit in a payload field, and every reorganisation rewrites it. Store the groups — roles, project ids, tenant ids, security labels — and resolve the asking user into their group set at query time.

This moves the explosion from the index to the query, where it is smaller but not free. A user who belongs to four hundred groups produces a four-hundred-term disjunction in the filter, and engines have limits on filter size and get slower well before they hit them. Three things keep it bounded:

• Resolve the group set once per request, not per chunk. It is the same set for every candidate; fetching it inside the scoring loop is the classic version of this bug.
• Collapse the hierarchy at write time. If membership in eng-platform implies read on everything under eng, index the closure rather than resolving the tree at query time. You pay for it on reorganisation, which is rare, instead of on every query, which is not.
• Scope by tenant first, then filter. A mandatory tenant predicate — or better, a per-tenant partition — cuts the candidate set by orders of magnitude before the group disjunction is evaluated, and it is the predicate you must never let a relevance threshold relax.

The multi-tenant half of this generalises beyond retrieval; multi-tenancy for agents treats the isolation question across the whole loop, and memory scopes and tenancy treats what happens when the agent's own writes carry the same labels.

Step 5Put the predicate somewhere application code cannot forget it.

Every design above depends on a filter argument being passed, which means it depends on no code path ever omitting it. That is a bad thing to depend on: the omission is one missing keyword argument, it produces better-looking results than the correct version, and no test that checks answer quality will catch it. The batch reindex job, the admin debugging tool and the eval harness are where the predicate goes missing, and all three run with credentials that can see everything.

Where you can, make the storage layer enforce it instead of asking callers to. Postgres row-level security with pgvector is the strongest commonly available version of this: the policy is attached to the table, so a query that forgets its WHERE clause returns the rows the current role may see rather than all of them, and the retrieval code physically cannot over-read. That is the reference monitor property — one place the request must pass through, which cannot be bypassed — and it is worth a measurable amount of query performance.

Where you cannot, the filter must be constructed by a single wrapper that takes the caller's identity as a required argument and has no code path that produces an unfiltered query. Make the unsafe version impossible to express rather than merely discouraged, and make the default fail closed: an identity you cannot resolve retrieves nothing, which is a visible outage and therefore gets fixed, rather than retrieving everything, which is a silent disclosure and therefore does not.

Step 6Two tests, because correctness and quality fail separately.

Permission-aware retrieval has two failure modes that no single number detects, so measure them apart.

The disclosure test is a correctness assertion and it should be boring. For each role in a fixture tenant, issue a set of queries deliberately written to be most similar to documents that role may not read, and assert that no forbidden document id appears in the results — before generation, at the retrieval boundary, where the assertion is cheap and exact. This is a unit test, it belongs in CI, and its expected value is zero forbidden hits forever. Add one case per incident you ever have.

The quality test is the one teams skip, and it is the one that catches the failure in STEP 1. Measure filtered recall@k for a given user against an oracle that is an exact search over only that user's permitted rows — not against the unfiltered index, which is a different question and flatters post-filtering. Run it for a low-v role, not an admin, because the admin is the one user for whom every implementation looks identical.

# the oracle that makes the number mean something

for role in [admin, dept_reader, project_member, contractor]:
    permitted = exact_scan(corpus.filter(role))      # ground truth
    got       = retrieve(query, k=10, identity=role)
    recall    = |got & permitted[:10]| / 10

# a post-filtering stack scores ~1.0 for admin and
# falls off a cliff as v shrinks. that shape IS the bug.

The general apparatus for the second test — separating retrieval quality from grounding from answer quality — is in evaluating RAG; what this page adds is that every one of those metrics must be computed per role, because a single aggregate figure is dominated by whichever role your eval set happens to use.