Candidate Generation: Many Sources, Not One

In one line: a single retriever encodes a single notion of relevance, and every production system runs several that disagree on purpose.

Why one source is never enough

The embedding-based retriever from the vector search chapter is excellent at one thing: finding items similar to what this user has engaged with. That is also its limitation.

The pattern: each source has a systematic blind spot, and the blind spots are different. Blending covers them.

The recency row matters more than it looks. A pure embedding retriever trained on historical engagement structurally cannot surface anything new, because a new item has no engagement to be similar to. Without a recency source, a catalogue slowly ossifies around whatever was popular when the model was trained — which is the feedback loop from the foundations module, arriving as an architectural gap rather than a training one.

Blending: quotas beat scores

The instinct is to score all candidates and take the top N. It does not work, for the same reason score fusion failed in hybrid search: the sources produce incomparable scores. A cosine similarity, a collaborative-filtering score and a recency rank are on different scales with different distributions.

Two approaches that do work.

Per-source quotas. Each source contributes a fixed allocation — 5,000 from embeddings, 2,000 collaborative, 1,000 recent, 500 popular. Crude, predictable, and it guarantees the exploratory sources are represented rather than being crowded out by the dominant one.

Rank fusion. Reciprocal rank fusion, exactly as in hybrid search: ignore the scores, use the positions, reward items several sources agree on.

In practice quotas are more common at this stage, and the reason is worth knowing: at candidate generation you are not trying to order anything — that is the ranker's job — you are trying to assemble a set with good coverage. Quotas give you coverage guarantees directly; fusion gives you an ordering you are about to throw away.

Deduplication is not optional

Sources overlap heavily — a popular item is likely also embedding-similar and collaboratively recommended. Without deduplication the same item occupies several candidate slots, which wastes the budget and can make the ranker see it repeatedly.

The subtlety is what counts as a duplicate. Exact item id is the easy case. Near-duplicates are the real problem: the same product from three sellers, the same article syndicated to four sites, three versions of one video. Those are distinct ids and identical to a user.

Deduplicating at the candidate stage is cheaper than at re-ranking, because you have not yet spent ranking compute on the copies. But it needs a notion of item identity beyond the id — a content hash, a cluster id, a canonical product key — which is a data-modelling decision made long before this pipeline exists.

Source failure and graceful degradation

Each source is a dependency, and treating them as independently failable is what separates a design from a diagram.

Two rules that follow.

Give each source its own timeout, and proceed without stragglers. Waiting for a slow source to protect recall costs latency on every request, and the ranker can usually work fine with a slightly smaller candidate set. A per-source deadline with partial results is almost always the right trade.

Keep a non-personalised floor. Popularity computed offline and cached is the fallback that always works, because it depends on nothing in the request path. It is much worse than the real system and enormously better than an empty page — the same argument as the non-ML fallback in the monitoring lesson.

Sizing the candidate set

How many candidates to generate is a real decision with a clean way to make it.

Too few and recall suffers — the ceiling from the previous lesson. Too many and the pre-ranker's cost rises linearly while the marginal candidate is increasingly unlikely to be good.

The empirical method: sweep the candidate count and measure recall of the finally-shown items. The curve flattens, and the useful setting is where it does. Going beyond the knee buys nothing and costs pre-ranking compute on every request.

Next: the stage between retrieval and ranking, and the way it fails.