Re-Ranking: Diversity, Policy, and the Set
In one line: the ranker produces an ordered list of individually good items, which is not the same thing as a good list.
Why a separate stage
A ranker scores each item independently, so its output can be ten near-identical results — each one correctly the best available answer, and collectively useless.
The properties that matter here are properties of the set:
| Property | Why per-item scoring cannot express it |
|---|---|
| Diversity | Depends on what else was selected |
| Freshness quota | "At least two new items" is a constraint on the list |
| Supplier or seller fairness | Nobody should own the whole page |
| Hard constraints | Out of stock, suspended, age-restricted |
| Slot rules | Position three is a sponsored slot |
| Deduplication | Near-identical items across sellers |
Each requires knowing what has already been chosen, which a function of one item does not.
Greedy selection with a penalty
The standard mechanism, and it is simpler than it sounds. Build the list one slot at a time. For each remaining candidate, score it as its relevance minus a penalty for how similar it is to what you have already picked. Take the best, add it, repeat.
score(item) = relevance(item) - lambda * max_similarity(item, already_selected)
lambda is the dial between relevance and diversity, and it is a product decision. Zero reproduces the ranker exactly; high values produce a varied and less relevant list.
Two things worth knowing about it.
It is greedy, not optimal. Selecting the globally best diverse set is combinatorial. Greedy is fast, predictable and good enough, and saying so is better than pretending it is optimal.
The similarity function is the real design choice. Similar by category, by embedding, by seller, by price band? Each produces a different kind of diversity, and the right one depends on what users experience as repetitive. Embedding similarity is the general answer and often too subtle — users notice the same seller five times more than they notice semantic closeness.
Constraints are filters, not penalties
The distinction from the ranking lesson, and this is where it is enforced.
A preference goes in a score: newer is better, higher-rated is better, cheaper is better. Scores trade off, and a strong enough signal elsewhere can outweigh them. That is correct behaviour for a preference.
A constraint must not be tradeable. Out-of-stock, suspended seller, age-restricted, region-blocked, legally excluded — no relevance score should be able to overcome these.
The ordering matters too. Filter before the expensive ranking where you can — scoring items you will discard is wasted compute — but some constraints depend on request-time state that is only fetched late, so a final filter pass is needed as well. Both, not either.
Freshness and exploration quotas
The mechanism that makes the exploration budget from the candidate lesson actually reach users.
Retrieving new items does nothing if the ranker never places them highly, and it will not — a new item has no engagement history, so every learned signal about it is weak. Exploration that is not protected at re-ranking is exploration that never happens.
So it is enforced as a slot quota: at least N of the visible positions go to items below some age or impression count.
The cost is direct and worth stating: those slots go to items expected to perform worse, so short-term engagement drops. That is the price of not letting the catalogue ossify, and it should be a deliberate, measured number rather than an accident.
Slot-aware and sequential effects
The refinement worth mentioning, since it is where this area is heading.
Everything above still treats positions as interchangeable given a diversity penalty. In reality what belongs at position one differs from what belongs at position eight — early slots reward confident relevance, later slots reward variety, because a user who scrolled that far did not find what they wanted.
Models that score a whole slate rather than items independently capture this. They are more expensive and only worth it when the list is genuinely consumed as a sequence — a feed, a carousel — rather than as a set of options.
Keep it inspectable
A closing property that matters operationally more than it sounds.
Re-ranking is where business rules live, and business rules change weekly. If they are entangled with a learned model, every policy change is a retraining cycle and nobody can answer "why is this item at position four?".
Keeping this stage explicit and rule-shaped — filters, quotas, a penalty with a readable parameter — means a policy question has an answer, and a policy change is a config change. That is worth some ranking quality, and it is the reason this stage stays deliberately simple in most production systems.
Key takeaway
Some properties belong to the list rather than to any item, and a per-item scorer structurally cannot express them — which is why re-ranking is a stage rather than a few more features. Greedy selection with a similarity penalty is the standard mechanism, and the similarity function matters more than the algorithm because users notice the same seller five times more than semantic closeness. Preferences go in scores and constraints go in filters, or a high enough relevance score will surface an out-of-stock item. Enforce exploration as a slot quota, since a new item will never win on learned signals. And keep the stage inspectable, because business rules change weekly and "why is this at position four?" needs an answer.
Next: serving all of this inside the budget.