Not a Smaller Web Search
In one line: enterprise search looks like an easier problem because the corpus is smaller, and it is harder, because almost every signal web search relies on is missing.
The three differences
The corpus belongs to somebody. Every document has an owner and an audience, and showing it to the wrong person is not a relevance failure — it is a data breach. Web search has no equivalent constraint; the pages are public by definition.
Nobody links to anything. PageRank works because the web is a graph of endorsements: a page many others link to is probably worth reading. An internal wiki has almost no link structure, and what exists carries no judgement — people link to what they happened to reference, not to what is authoritative.
There are three thousand users, not three hundred million. Web search learns from billions of interactions a day. An internal tool might see a few thousand queries, most of them unique. There is not enough signal to train a ranker, and not enough traffic to A/B test one.
Each of those removes a tool rather than shrinking the problem. That is why "it is only a few hundred thousand documents" is the wrong first reaction.
What replaces them
The design has to manufacture substitutes, and knowing what each one replaces is the shape of the whole chapter.
| Missing | Substitute |
|---|---|
| Public corpus | Permissions in the retrieval predicate |
| Link-based authority | Recency, author role, canonical status, usage |
| Click-trained ranking | Hand-tuned signals plus a judged evaluation set |
| Crawlable pages | Connectors to a dozen systems, each with its own API |
The fourth row is where the engineering time actually goes, and it is invisible in most designs.
The failure that defines the product
Enterprise search has one failure mode that is qualitatively worse than everything else: showing someone a document they are not entitled to see.
It is not a bad result. It is a compliance incident, and it is often unrecoverable — the person read it, and there is no undo. The person who ran the query did nothing wrong, so the failure is entirely the system's.
Two consequences follow, and both are architectural.
Permissions cannot be a filter applied after retrieval. Result counts, facet counts, snippets and "did you mean" suggestions all leak the existence of documents a post-filter would remove. The permission check belongs inside the retrieval predicate.
It is not enough to be right; you must be able to show you were right. Somebody will ask what a specific person could see on a specific date, and "the system filtered correctly" is not an answer. The audit trail is a first-class requirement rather than a logging detail.
Where the value is
The counterweight, and it is worth stating so the design does not become purely defensive.
The reason to build this is that the knowledge exists and nobody can find it. A new joiner asks a question in a chat channel that was answered in a document eighteen months ago, and the answer to "does our expense policy cover this" is somewhere in a Confluence page nobody remembers.
Which points at the actual objective: not finding documents, but answering questions. That is the Q&A half, and it changes the bar. A search result list transfers the reading to the user. An answer takes responsibility for being right — and being confidently wrong about a policy is worse than returning nothing.
Scoping
Four questions before any architecture.
Which sources, and how many? This decides most of the effort. Three systems is a project; fifteen is a programme, and each one has its own auth, rate limits, formats and change semantics.
How sensitive is the most sensitive thing indexed? Public-by-default internal documentation is a different risk posture from a corpus containing HR files, legal matters and unannounced financials. The second forces stricter permission handling everywhere.
Search, or answers? Returning ranked documents and generating a grounded answer are different products with different failure modes and different acceptance bars.
How many users, and how much do they query? This decides whether any learned ranking is possible at all, and it is usually no.
Committing
An internal assistant over about a million documents from eight systems — Drive, Confluence, Slack, Jira, a ticketing system, an HR system, a code host and email. Around five thousand employees, a few thousand queries a day. Almost everything is permissioned, and some of it is highly sensitive. We want grounded answers with citations, not just a result list. And there is no click data worth training on, so ranking will be hand-tuned signals evaluated against a judged set.
Every clause is used later. Eight systems means connectors dominate. Highly sensitive content means the permission model is the centre of the design. Answers rather than results means abstention matters. And no click data means the evaluation strategy is the hard part rather than an afterthought.
Key takeaway
Enterprise search removes three of web search's foundations: the corpus is owned, so a wrong result is a breach rather than a miss; nothing links to anything, so no signal confers authority; and there are thousands of users, so no ranker can be trained. Each has to be substituted deliberately — permissions in the retrieval predicate, manufactured authority signals, and a judged evaluation set — and the connectors nobody draws are where the engineering time actually goes.
Next: those connectors.