Free preview

Caching: Three Tiers

In one line: three different caches sit at three different points, they save different things, and only one of them can return an answer to a question nobody asked.

The three, and what each saves

Exact cachePrefix cacheSemantic cache
Keys onThe full request, byte-identicalA shared token prefixEmbedding similarity to a past request
SavesThe entire generationPrefill only, not decodeThe entire generation
LivesIn front of the modelInside the engine, in GPU memoryIn front of the model
Hit rateLow — exact repeats are rareVery high — system prompts repeat alwaysModerate to high
RiskStaleness onlyNone — it is a pure optimisationReturning the wrong answer confidently

The columns are ordered by how much you should trust them, and that ordering is the lesson.

Prefix caching is the free one

Every request carries a system prompt, and in a retrieval system it carries retrieved context too. Those tokens are processed on every request even though the model has seen them a thousand times.

Prefix caching keeps the processed KV state for a shared prefix and reuses it, so only the new suffix is prefilled.

Two design consequences follow, and both are cheap to state and easy to miss.

**Put the stable part of the prompt first. A cache keyed on a prefix only matches from the beginning, so a request that starts with a per-user timestamp shares nothing with the next request. Ordering the prompt stable-to-variable is a one-line change with a large effect.

It only helps prefill. Decode still runs in full. So prefix caching improves time to first token and throughput, and does nothing for a request whose cost is dominated by a long generation.

This is also the mechanism that makes prefix-aware routing worth doing — the cache exists per worker, so sending a request to the wrong worker discards a hit that was available.

Exact caching is safe and rarely hits

Key on a hash of the complete request; return the stored response on a match. Trivially correct, and in a conversational product almost nothing repeats byte-for-byte.

It earns its place in narrower cases: a fixed set of suggested prompts, an autocomplete over a bounded vocabulary, an internal pipeline that reprocesses the same documents, or an evaluation harness that runs the same suite repeatedly. That last one is worth remembering — evaluation runs are real inference spend, and they are highly cacheable.

Semantic caching is powerful and dangerous

Embed the incoming request, search for a past request within a similarity threshold, and return its stored answer.

The hit rate is far higher than exact caching, because paraphrases are common. And the failure mode is the sharpest in this chapter: two requests can be extremely close in embedding space and have different correct answers.**

This is the first lesson of the retrieval chapter arriving with consequences: embeddings encode topic strongly and specifics weakly, so the distinguishing detail — a date, a number, an identifier, a negation — is exactly what the similarity score is worst at seeing.

Making it safe

The mitigations are all about narrowing where it applies rather than tuning the threshold alone.

**Scope the key. Include user, tenant, locale and anything else that changes the correct answer, so a hit can only come from a genuinely comparable context.

Do not cache anything personalised or time-sensitive. If the answer depends on who is asking or when, semantic caching is unsafe by construction.

Set the threshold empirically, and high. Take labelled pairs, measure how often a hit at a given similarity would have returned a wrong answer, and choose accordingly. A conservative threshold with a lower hit rate is the right default.

Measure the harm, not the hit rate.** The tempting metric is cache hit rate, and it rewards exactly the wrong behaviour — loosening the threshold always improves it. Sample hits and check whether the served answer was actually correct for the new question.

Invalidation

The usual hard problem, with an extra wrinkle: a cached answer can go stale because the world changed, not because anything in your system did.

Three mechanisms, and most systems need more than one:

**Time to live. Blunt and effective. Short for anything current, long for stable reference material.

Event-driven invalidation. When a source document changes, drop the cached answers derived from it. This requires tracking which retrieved chunks contributed to which cached response — worth doing in a retrieval system, and it is provenance you probably want anyway.

Version keying.** Include the model version, prompt version and index version in the cache key. Then a deploy invalidates cleanly and automatically rather than serving answers from the previous configuration.

That third one is the one teams forget, and its symptom is confusing: a prompt improvement ships and quality does not move, because most traffic is being served from a cache populated by the old prompt.

Key takeaway

Three caches at three points: prefix caching is a pure optimisation inside the engine that saves prefill only, so put the stable part of the prompt first; exact caching is safe and rarely hits outside bounded or repeated workloads like evaluation runs; and semantic caching has the highest return and can serve a confidently wrong answer, because embeddings see topic clearly and the distinguishing detail poorly. Scope its key, exclude anything personalised or time-sensitive, and measure served-answer correctness rather than hit rate — which is a metric that rewards loosening. Version the cache key on model, prompt and index, or a deploy will keep serving the old configuration.

Next: choosing which model handles the request at all.

Enjoying the preview?

Create a free account to unlock the rest of this course, the in-browser judge, and live AI mock interviews.

Sign up free to continue