Early and Late Binding

In one line: you can store who may see a document when you index it, or ask the source when someone searches, and each is wrong in a different way.

The two models

Early binding — also called index-time security. The connector reads the ACL alongside the content and stores it in the index. A query carries the user's permission set and intersects it with each document's, inside the search itself.

Late binding — query-time security. The index holds content and identifiers. After retrieving candidates, the system checks each one against the source system, or against a live permission service, and drops what the user cannot see.

What each gets wrong

Early binding is fast and stale. The intersection happens inside the query, so it costs almost nothing and it scales — and the ACL in the index is a snapshot from the last sync. Between syncs it can be wrong in both directions: someone who lost access still has it, someone who gained access does not yet.

The first is a security problem, the second is merely annoying, and that asymmetry drives most of the mitigation.

Late binding is correct and expensive. It asks the authority, so it is right at the moment of the query. And it means a call per candidate document, to a system with its own rate limits and availability — which turns every search into a fan-out against sources you do not control.

There is a second problem people miss: late binding breaks counting. You cannot know how many results a user is entitled to without checking all of them, so result totals, facet counts and pagination all become approximations. Retrieve fifty, discard forty-eight after checking, and you must fetch more and check again to fill a page.

The hybrid

What production systems actually do, and it is the answer to give.

Early binding for the retrieval predicate. The index holds ACLs and does the intersection inside the query, so retrieval is fast and the candidate set is already almost right.

Late binding as a verification step, on the small number of results actually being returned — ten, not a thousand. One check per displayed result is affordable; one per candidate is not.

Tiered by sensitivity. Ordinary content is served on the index's word. The most sensitive sources verify every time, accepting the latency because the cost of being wrong is categorically higher there.

That divergence log is worth proposing explicitly. Every time verification removes a result the index thought was permitted, the index was stale — so the rate of those events is a direct measure of your permission freshness, and it is the only honest one available.

Representing an ACL in an index

The practical mechanics, since "store the ACL" hides real decisions.

The standard shape is two sets per document: allow and deny, each containing principal identifiers — user ids, group ids, role ids. The query carries the user's expanded principal set and the predicate is a set intersection with allow, minus any intersection with deny.

Three details that matter.

Deny must be evaluated after allow, because deny normally overrides. A model with only allow-lists cannot represent "everyone in Engineering except contractors", which is a common real rule.

Store group identifiers, not expanded membership. Expanding groups into individual users at index time means every membership change requires re-indexing every affected document — which for a large group is the entire corpus. Store the group id; expand the user's side at query time instead, where it is one small operation.

The identifiers must be stable and normalised. Eight source systems use eight identity schemes, and the same person appears as an email in one, a directory id in another, a username in a third. Normalising to a canonical principal id is a prerequisite for the intersection meaning anything, and it is a genuine piece of work.

That second point is the one worth saying out loud. It inverts where the expansion happens — per query rather than per document — and it is the difference between a workable system and one that re-indexes constantly.

Chunk-level permissions

A wrinkle specific to Q&A rather than search.

Retrieval operates on chunks, and permissions are defined on documents. So the ACL has to propagate from document to chunk, and every chunk must carry it — because a chunk retrieved without its ACL is a permission bypass hiding inside a normal-looking pipeline.

Two failure modes worth naming. A chunk that inherits the wrong document's ACL after a re-index that changed boundaries. And a cached embedding or a cached chunk that outlives a permission change.

The rule: the ACL travels with the chunk, and it is re-checked at retrieval rather than trusted from the cache.

When neither is needed

The judgement to show.

If every document is visible to everyone — a public documentation site, an internal handbook with no restrictions — none of this applies, and building it is real complexity for no benefit.

So the first question is what fraction of the corpus is actually restricted. If it is nearly none, index it flat and spend the effort elsewhere. If it is nearly all, the permission model is the design and everything else is subordinate to it.

Next: turning a person into a permission set.