Head, Torso and Tail

In one line: a small number of queries account for most of your traffic, and the majority of distinct queries have been seen once or never — so one strategy cannot serve both ends.

The shape

Search query distributions are heavily skewed. A small fraction of distinct queries covers the large majority of traffic; the remaining distinct queries are individually rare, many appearing once ever.

Both halves of that are load-bearing, and candidates usually remember only the first.

By traffic, the head dominates — so improving head queries moves your aggregate metrics.

By distinct query, the tail dominates — so most of the queries a user could type are ones you have no data for. And tail queries carry high intent: someone typing eleven words knows precisely what they want.

Three segments, three strategies

Head — curate and cache

You have abundant data per query. Click-through, conversion and dwell are all statistically meaningful for a query seen ten thousand times a day, so you can learn a per-query ordering directly rather than generalising to it.

Which means the head can be curated. Merchandising rules, editorially pinned results, and per-query learned orderings are all viable and all beat a general model on these queries specifically.

It also caches beautifully. The same query returns the same result set for most users, so the full pipeline runs once and serves many. That is a large latency and cost win, and the invalidation question — how stale can these results be — is a product decision that differs by vertical.

The trap: over-fitting your evaluation to the head. Sampling queries by traffic to build an eval set produces a set that is nearly all head queries, so a model that regresses badly on the tail scores fine.

Torso — where models earn their keep

Enough traffic to have some signal, not enough for per-query treatment. This is exactly where a learned ranker generalising across similar queries beats both curation and pure BM25, and it is the segment where model improvements show up most clearly.

Tail — generalise, or fail

No per-query data, ever. Nothing to learn from, nothing to curate, and the queries are long and specific.

Four things carry the tail, and they are all about generalisation rather than memorisation:

Semantic retrieval, because a long conversational query has little term overlap with a document written in different words.

Query relaxation, because a nine-term query often has no document containing all nine, and dropping the least informative terms finds the answer.

Entity extraction, because a long query is usually several constraints and a topic — and once decomposed, it stops being rare. waterproof hiking boots size 11 wide fit under £120 is a unique string and an entirely ordinary combination of attributes.

Robust defaults, because a tail query with a mediocre answer is better than a tail query with nothing.

That third point is the one worth carrying into an interview: decomposition converts a tail query into a head query in attribute space. The string is unique; the constraint set is not.

The evaluation consequence

The trap that catches teams, stated plainly.

Sample your evaluation queries by traffic and you get a head-weighted set. Every improvement targeting the head looks excellent, tail regressions are invisible, and the reported metric rises while the experience for specific asks degrades.

The fix is to stratify: sample separately from head, torso and tail, report each segment separately, and weight the aggregate deliberately rather than letting the traffic distribution weight it for you.

Latency differs too

An under-discussed practical point.

Head queries are cached, so they are fast. Tail queries run the full pipeline — expansion, hybrid retrieval, the ranker — and are slow. So your p50 latency reflects head queries and your p99 reflects the tail.

Which means p99 latency and tail query quality are the same conversation. Optimising p50 by improving cache hit rates does nothing for the queries that are already slowest, and the users experiencing p99 are the ones with the most specific intent.

Where to spend

A useful framing for the inevitable prioritisation question.

Head improvements move the aggregate metric and are usually already well-served — the marginal gain is small because a lot of effort has gone there. Tail improvements barely move the aggregate and fix genuine failures for users with high intent, which correlates with conversion and with whether they come back.

Neither is right in general. The point is to be explicit about which you are buying, because a team that only ever measures the aggregate will always choose the head, without ever deciding to.

Next: the queries that return nothing at all.