Concept Drills: 15 Distributed Cache Probes

Cover the answer, attempt it out loud, then compare. If your answer is a definition rather than a decision, it is not yet an interview answer.

1. Why put a cache in RAM rather than on fast SSD?

Weak: "RAM is faster."

Strong: "It's an order-of-magnitude question, not a preference. In the memory hierarchy — CPU register, CPU cache, RAM, SSD, magnetic disk — RAM is the layer where access is fast enough to make a hit meaningfully cheaper than the database call it replaces, and large enough to hold a useful working set. Going to SSD narrows the gap between hit and miss until the cache stops earning its complexity. The whole design assumes we serve from RAM; that assumption is why the cache is volatile and why it can be rebuilt from scratch after a crash."

2. A system writes data and reads it back promptly, and needs cache and database consistent. Which writing policy?

Weak: "Write-back, it's fastest."

Strong: "Write-through. It's the only one that satisfies both halves. Write-back leaves a window where the database is behind, and can lose the write entirely if the cache dies before flushing. Write-around guarantees the immediate read-back is a miss — precisely the opposite of what was asked. Write-through puts the data in the cache the moment the write completes and keeps both stores in agreement. You pay for it in write latency, and that's the trade being made."

3. What exactly does write-back risk?

Weak: "Some inconsistency."

Strong: "It can lose acknowledged writes. The client is told the write succeeded once it's in the cache — which is volatile RAM. If the node dies before the asynchronous flush, that write is gone with no record it ever existed. That's not a corner case to note in passing; it means choosing write-back is accepting data loss as a design position. Fine for a view counter, wrong for anything the business would miss."

4. Eviction and invalidation — what's the difference?

Weak: "Both remove entries from the cache."

Strong: "Different questions. Eviction asks what should I drop to make room? — triggered by the cache being full, governed by LRU or LFU or FIFO. The evicted entry was still correct, just least valuable. Invalidation asks what is no longer true? — triggered by the data going stale, governed by TTL or explicit purge. The invalidated entry was wrong. Not doing eviction means you can't admit new entries; not doing invalidation means you serve incorrect data. Conflating them produces designs that assume a full cache is also a fresh one."

5. Active or passive expiration?

Weak: "Passive — no background cost."

Strong: "Both, in practice. Passive checks TTL only when a key is requested, which is free but leaks memory — an entry nobody requests is never checked and sits expired forever. Active runs a background sweep, which reclaims that memory but burns CPU scanning entries nobody will ask for. Running both gives you passive catching expiry exactly when it matters and an active sweep reclaiming what nothing is asking about."

6. Why LRU rather than LFU for a social feed?

Weak: "LRU is standard."

Strong: "Because LRU encodes locality of reference, and social content has strong recency locality — recent posts get the views. LFU keeps whatever has the highest total access count, so an item that was enormously popular last year can squat a cache slot forever, outranking something popular right now. That's cache pollution: the hit rate degrades while the policy behaves exactly as specified. Real LFU implementations add aging or windowing to decay old counts."

7. Where is hashing used in this design, and how many times?

Weak: "To pick a server."

Strong: "Twice, at two different levels. Consistent hashing in the cache client selects which server holds a key — O(log N) in the number of shards, and minimizes rehashing when servers are added or removed. Then inside that server, a plain hash table locates the value in O(1) average time. People conflate these; they're separate mechanisms solving separate problems, and only the first one needs to be consistent."

8. Why a hash map and a doubly linked list?

Weak: "One for lookup, one for ordering."

Strong: "Because either alone fails one of the two operations. The hash map gives O(1) lookup but has no notion of order, so it can't tell you what to evict. A doubly linked list maintains access order and supports O(1) removal and reinsertion at either end — but finding a specific node in it is O(n). Together: the map holds pointers into the list, so you locate a node in O(1) and then splice it to the head in O(1). Both operations constant time, which is what makes LRU practical at all."

9. Dedicated cache servers or co-located with the application?

Weak: "Co-located — it's cheaper."

Strong: "Depends on whether the two tiers scale together, and they usually don't. Traffic growth adds application load; working-set growth adds cache pressure, and those move independently. Co-location forces you to buy both. It also compounds on failure: losing a host loses an app instance and its cache partition, so the survivors take more traffic and see a lower hit rate — the two effects multiply. I'd default to dedicated servers, which also lets you pick different hardware per tier and enables cache-as-a-service — with key namespacing so two services caching user:42 don't collide."

10. What's the one invariant the cache client must maintain?

Weak: "It needs to know the servers."

Strong: "Every client must use the same hashing algorithm and the same server list, so a PUT and a GET for the same key reach the same server. And the failure mode when they don't is the point: nothing errors. The write lands on server A, the read goes to server B and misses. The cache just quietly stops working for the affected keys — the only symptom is a collapsing hit rate and rising database load. That silent failure is why a config file isn't good enough and a configuration service is."

11. Why is a configuration service worth its complexity over a config file?

Weak: "It's more robust."

Strong: "Because both file-based options require a human to notice a server died and push an update. These failures happen on machine timescales, and every second of the gap is a client routing to a dead node. The configuration service continuously monitors health and notifies clients automatically — removing the human from a loop that operates faster than humans do. Same argument as four-nines availability requiring automated recovery: once your budget is minutes, a person in the path has already spent it. It also earns its keep twice, since it's the natural place to detect primary failure and select a new leader."

12. Given a 5 ms hit and 30 ms miss at p99.9, what is LRU at a 5% miss rate worth versus MFU at 10%?

Weak: "LRU is better."

Strong: "Run the numbers. Effective access time is hit ratio times hit time plus miss ratio times miss time.

MFU:  0.90 x 5  + 0.10 x 30  = 4.5  + 3.0 = 7.5 ms
LRU:  0.95 x 5  + 0.05 x 30  = 4.75 + 1.5 = 6.25 ms

1.25 ms, or about 17%, from the eviction policy alone. The leverage comes from the 6x gap between hit and miss cost — so this matters more the more expensive a miss is, and barely at all when a miss is cheap. Note both figures are p99.9, not averages: using means would understate the miss cost badly, because misses are the slow requests."

13. One key is receiving enormous traffic. Fix it.

Weak: "Add more shards."

Strong: "Sharding can't fix this, and saying so is the answer. Sharding splits key ranges, and a single key is the atom of that scheme — no amount of resharding helps. Replicas give a fixed 3x, not a scalable multiple. Beyond that: client-side dynamic replication, where clients replicate the hot key to extra servers and spread reads, or split the key artificially into key:1 through key:N with the client picking one and recombining. That's real application-level work. The framing: sharding spreads keys evenly, but traffic follows popularity, not key distribution."

14. Why do caches accept eventual consistency across data centers?

Weak: "CAP theorem."

Strong: "PACELC, and specifically the else branch. CAP covers the partition case — during a partition we pick availability, easy to justify since a stale entry is recoverable while an unavailable cache dumps 20x load on the database. But the decision here isn't driven by partitions: even with no partition, synchronous cross-region writes cost too much latency, so we choose latency over consistency permanently. That's why synchronous replication is fine 'when nodes are in close proximity' and untenable otherwise. Related rule: a node rejoining after an outage must not serve until it's fully synchronized — a dead node produces misses, which are correct-if-slow; a stale node produces confident hits, which are fast and wrong."

15. Memcached or Redis?

Weak: "Redis — it has more features."

Strong: "Requirements decide. Memcached is shared-nothing — servers never talk to each other, the client does routing and hashing, throughput is deterministic O(1), and it's multithreaded. It's what this design is: client-side hashing, internal hash tables, LRU, no server-to-server communication. Facebook has run roughly this since 2004 — about 28 TB across 800+ servers at a 95% hit rate.

Redis is a data structure store: sorted sets, hash maps, bitmaps, persistence via AOF and RDB, replication and clustering built in, transactions, Lua. The costs are complexity, single-threaded execution, and one thing worth stating plainly — replication is asynchronous, so a failover can lose an acknowledged write. Acceptable for a cache; genuinely dangerous once someone uses Redis as a database, which Redis invites.

Either way, pipeline: it's 5x throughput even on loopback, which tells you the bottleneck is per-request syscall overhead, not the network."

Read-the-diagram drills

15. One node of ten dies. How much of the cache is lost?

Strong: "Almost all of it — around 90%, not 10%. Changing the modulus changes the mapping for nearly every key, so keys that were fine on surviving nodes now hash somewhere else and miss.

That's the failure worth naming precisely, because the intuitive answer is 'we lost a tenth of our capacity'. What actually happens is a near-total cache wipe, and the resulting miss storm lands on the database at exactly the moment we've also lost a node of capacity.

Consistent hashing bounds the damage to the departed node's own arc — only those keys move. And virtual nodes on top, because with a handful of servers the ring is lumpy and one server ends up owning a disproportionate share."

16. Why does this graph spike, and why is it worse for a good cache?

Strong: "Thundering herd. The cache was absorbing every read for that key; the instant it expires, all of those in-flight requests become database queries simultaneously.

And the perverse part: the spike is proportional to how well the cache was working. A key served 10,000 reads/sec from cache produces a 10,000-query burst the moment it expires. Effectiveness and blast radius are the same number.

Three fixes that compose. Request coalescing is the strongest — the first miss fetches and every concurrent miss waits on it, so origin load is capped at one query per key no matter the arrival rate. Probabilistic early expiry staggers refreshes so the synchronized cliff never forms. Serve stale while revalidating returns the old value and refreshes behind it.

It's the same shape as synchronized retries after an outage: stagger the timing, or elect one actor to do the work for everyone."

17. Will adding replicas fix both of these?

Strong: "Reads yes, writes no.

For hot reads, replicas genuinely help — reads spread across primary and secondaries. But it's a fixed multiple, typically around 3×, not something that scales. Past that you replicate the key itself under suffixed names and have clients pick one at random, which is real application-code work rather than a config change.

For hot writes, replicas do nothing at all: every replica has to apply every write, so you've multiplied the work rather than divided it.

The fix there is client-side batching — buffer for 50–100ms and apply one consolidated update instead of thousands. That works when only the final value matters, like a view counter, and fails when every individual event must be observable.

Worth adding what doesn't work for either: resharding. Sharding splits key ranges, and a single key is the atom of the partitioning scheme — it lives on one node however finely you slice the ring."