High-Level Design
In one line: this is the design you would draw in the first two minutes of an interview. Knowing exactly what it leaves unsolved is what lets you evolve it under questioning rather than defend it.
The design
The two components
| Component | Responsibility |
|---|---|
| Cache client | A library residing in the service application servers. Maintains metadata about cache servers, and for every insert or retrieve selects a server using a hashing algorithm. All clients must share a consistent view of the server list and hashing strategy so requests for the same data route to the same server |
| Cache servers | Store the cached data. Each server is accessible by all clients and connects to the database. Clients communicate via TCP or UDP. If a server fails, clients treat requests to it as cache misses |
Note where the routing logic lives
The client is a library in your application process, not a proxy the request passes through. That means selecting a cache server costs a hash computation — no extra network hop, no separate tier to run and scale.
This is client-side load balancing, exactly as load balancing described it: lower latency because there is no intermediary, at the cost of putting logic in every client and needing all of them to agree. The rest of this chapter is largely about managing that second cost.
What this design cannot do
Three specific limitations, which the detailed design addresses:
| Limitation | The problem |
|---|---|
| Service discovery | Cache clients have no mechanism to detect when cache servers are added or fail |
| SPOF and performance | Using a single server for a dataset creates a SPOF. Additionally, frequently accessed data (hotkeys) can overload a single node, degrading performance |
| Server internals | The design lacks details regarding internal data structures and eviction policies |
The first limitation is the one that bites soonest
Lesson 5 established that all clients must share the same server list, or writes and reads for a key land on different servers and the cache silently stops working.
This design gives clients that list — and no way to update it. Add a server and clients do not know. Lose a server and clients keep routing to it. The correctness invariant depends on something the architecture has no mechanism to maintain.
Raising this yourself, before the interviewer does, is the natural way to move into the detailed design.
Hotkeys are the limitation people forget
Sharding spreads keys evenly. It does not spread traffic evenly, because traffic follows popularity rather than key distribution.
A single extremely popular key — a celebrity profile, a homepage config, a trending item — lives on exactly one shard, and no amount of resharding helps, because the key cannot be split. That shard becomes a hotspot while its peers idle.
Same shape as the databases material's warning that even key distribution is not even load. Lesson 8's replicas are part of the answer; Lesson 10 covers the rest.
Key takeaway
A client library that hashes, and servers that store. Simple enough to draw immediately — and it has no way to keep clients in sync, no redundancy, and no defined internals. Those three gaps are the next lesson.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "Application servers talk to a cluster of cache servers." |
| L5 | Places the logic: "the cache client is a library in the app servers that hashes the key to pick a server — no extra hop." |
| Staff+ | Volunteers the gaps: "this needs three things it doesn't have — a way to keep every client's server list in sync, replication so one node isn't a SPOF, and a hotkey story, because sharding spreads keys evenly but traffic follows popularity." |
Next: fixing all three.