Query Understanding

In one line: a sequence of cheap transformations turns two words into a structured request, and each one can improve recall or quietly destroy the query.

The pipeline

Order matters, because each stage consumes the previous stage's output.

Normalisation and analysis

Unglamorous and load-bearing. Lowercase, strip accents, normalise unicode, decide what to do with punctuation and whether hyphens split.

The rule that matters more than any individual choice: the query and the documents must be analysed identically. A query stemmed with one analyser against an index built with another will silently fail to match, and it fails partially — some terms match and some do not, so results appear rather than an obvious error. It is the single most common cause of "search is broken and I cannot see why".

Stemming itself is a trade. Reducing running to run improves recall and destroys precision where the inflection carries meaning. Aggressive stemmers conflate university and universe. The usual answer is a light stemmer plus explicit synonyms for the cases you care about.

Spelling correction

Roughly a tenth of queries in consumer search contain a typo, and an uncorrected typo usually means zero results.

Two mechanisms, and both are needed. Edit distance against the index vocabulary catches ordinary typos and is cheap. Query-log-based correction catches the rest: users who typed a misspelling and then immediately typed the correct form have labelled the pair for you, at scale, for free.

The design decision worth stating is what to do with the correction. Silently correcting is fast and occasionally wrong in a way the user cannot see. Better is to correct and say so — "showing results for X, search instead for Y" — which is recoverable. And a correction should never fire when the original returns good results: a real product name that resembles a common word gets destroyed by an over-eager corrector.

Entity and attribute extraction

The stage that carries e-commerce and gets skipped in most interview answers.

red nike running shoes size 10 under £80 is not a bag of words. It is: colour red, brand nike, category running shoes, size 10, price ceiling 80. Extracting that turns a fuzzy text match into a filtered retrieval over structured fields, which is both more accurate and much faster.

The distinction to hold: an extracted attribute is a hard constraint, and a topical term is a soft one. Size 10 is not negotiable — a size 9 result is wrong, not slightly worse. running is negotiable; a cross-trainer might be fine.

Collapsing that distinction is the classic e-commerce search failure: everything becomes a soft text match, and the top result is a beautiful size 7 shoe.

Intent classification

Cheap and it changes the result set completely.

Navigational queries want one specific destination and should return it first with high confidence — a brand name query should not return a category page above the brand. Informational queries want breadth. Transactional queries want items that can be acted on, which is where availability and price enter ranking.

In e-commerce the useful axis is broad-versus-specific. shoes is a browse intent and should return a diverse category view. nike air max 90 white size 10 is a known-item lookup and should return that item, then near-misses. Applying the same ranking to both makes browse queries feel arbitrary and specific queries feel unhelpful.

Expansion and rewriting

The highest-leverage and highest-risk stage.

Synonym expansion — laptop also matches notebook. Improves recall and needs to be directional: notebook should not always expand to laptop, because some people mean paper.

Query relaxation — drop the least important term when the full query returns too little. Powerful and the mechanism behind zero-result recovery.

Learned rewriting — mine reformulation chains from the logs. A user who searched X, got nothing useful and then searched Y has told you that X means Y. This is the cheapest high-quality rewriting data available and it requires no labelling.

That gate is the design detail worth stating. Expansion should fire on low-recall queries and stay out of the way otherwise, which makes it a fallback rather than a default.

The cost

All of this sits in front of retrieval, so it is on the critical path. Normalisation and analysis are microseconds. Dictionary lookups for spelling and synonyms are sub-millisecond. Entity extraction with a small model is a few milliseconds. An LLM-based rewrite is hundreds — which is why it belongs offline, precomputed for head queries, rather than in the request path.

That split is the practical answer: the head of the query distribution can have expensive understanding computed ahead of time and cached, and the tail gets the cheap online pipeline.

Next: the inverted index, and why BM25 has survived everything.