WebSockets and the Connection Registry

In one line: the Twitter and Uber chapters both needed a way to find which server holds a given user's connection, and neither included one. This chapter finally builds it.

Why WebSockets

HTTP(S) doesn't keep the connection open for servers to send frequent data to a client. With HTTP, a client constantly requests updates — polling — which is resource-intensive and causes latency. WebSocket maintains a persistent connection, transferring data immediately whenever it becomes available. It provides a bidirectional connection.

The requirement is server-initiated delivery, and only one thing satisfies it

The reason is not that WebSockets are faster. It is that the server needs to initiate.

A message arrives for you when someone else sends it — an event the client cannot predict and therefore cannot request. HTTP's request-response model has no way to express that:

Polling's cost is worth quantifying against Lesson 3's numbers. With 2 billion users polling every 5 seconds:

2,000,000,000 / 5 = 400 MILLION requests per second

That is 345 times the actual message rate of 1.16 million per second — and the overwhelming majority return nothing. The Uber and web-crawler chapters both met this shape: polling asks whether anything happened; pushing tells you when it did.

When events originate at the server and the client cannot predict them, a persistent bidirectional connection is not an optimization — it is the only structure that works.

The connection registry

Each active device establishes a WebSocket connection. WebSocket servers maintain persistent connections with online users, distributed across a cluster. Each server maintains a session per active connection. A WebSocket manager maintains the mapping between users, active connections, and servers, stored in a Redis cluster.

This is the component Twitter and Uber both needed and neither drew

Worth stating plainly, because it closes a gap this module has left open twice.

The Twitter chapter established that ride offers — sorry, that timeline pushes and notifications — are server-initiated and must reach one specific user's socket. It never showed a registry.

The Uber chapter was sharper: a ride offer must reach one specific driver's connection, which lives on one specific server, and the design had no component that knew which. I flagged it as a real gap.

Here it exists: the WebSocket manager, backed by Redis.

The properties it needs are the coordination-store properties from that building block's ZooKeeper discussion:

That last row is what makes it harder than ZooKeeper's usual job. Cluster topology changes on deploys — rarely. A connection registry for 2 billion users changes constantly, as people open and close apps, lose signal, and switch networks.

Which is why Redis rather than a consensus store: at that churn rate, a consensus protocol per update would be ruinous. You accept weaker consistency and handle the failure mode instead.

A registry that changes as often as user behaviour cannot be a consensus store; it has to be a fast cache with a repair path.

Detecting a connection that is already dead

A WebSocket can be functionally dead long before either side notices — a phone loses signal, a NAT entry expires, a laptop sleeps. The socket looks open and nothing arrives.

Application-level heartbeats are what make liveness knowable. TCP alone will not tell you in useful time — its own keepalives operate on a scale of hours. A ping every ten to thirty seconds bounds detection to roughly one interval.

Per-chat sequence numbers solve the other half. Each message carries a monotonically increasing number within its chat, so a client that receives 7 after 5 knows it missed 6 and can request a sync immediately rather than waiting to notice. Piggybacking the current sequence on the heartbeat means a client detects a gap even during a quiet period with no traffic.

Routing a message to the right server

With millions of connections spread over many hosts, the sender's server almost never holds the recipient's connection.

Both work. Pub/sub with a registry is usually preferred because it avoids an all-to-all connection mesh between chat servers and tolerates membership changes gracefully. Consistent hashing makes routing computable without a lookup, at the cost of that mesh and of reshuffling users whenever the ring changes.

Whichever you pick, the pub/sub delivery is best-effort — and that is acceptable precisely because the inbox already guarantees eventual delivery.

Caching the registry

Both sender and receiver rely on the manager to locate each other's server. During an ongoing conversation this can result in frequent lookups. To reduce latency, each WebSocket server caches:

• If both users are on the same server, the manager call is avoided entirely.
• Information about recent conversations — which user is on which server.

Caching the registry works because conversations are bursty

The lookup is on the path of every message, so at Lesson 3's 1.16 million messages per second it would be 1.16 million Redis reads per second before caching.

Caching helps enormously here for a workload-specific reason: conversations are bursty and repetitive. People exchange many messages with the same person over minutes, and both endpoints stay put for the duration.

First message in a conversation -> manager lookup
Next 50 messages                -> cache hit

So the hit rate on an active conversation approaches one, and the manager sees roughly one lookup per conversation rather than per message.

The same-server shortcut is the stronger optimization though. With 200 servers, two random users share a server about 0.5% of the time — but conversation partners are not random. Regional routing means people who talk to each other are often on the same regional cluster, so the real hit rate is far above chance.

Cache a lookup when the access pattern is repetitive, and the burstiness of human conversation makes this one of the most cacheable lookups in the module.

The invalidation answer is right, and it has a race the design does not name

The manager updates the mapping when a user disconnects and reconnects to a different server. It then invalidates the cached entries on the WebSocket servers, and the updated mapping is propagated to the relevant caches. Cached entries remain valid until the manager sends an invalidation signal.

Push-based invalidation rather than TTL expiry, which is right: a TTL long enough to be useful is long enough to misroute many messages, and one short enough to be safe defeats the cache.

But there is a window the description does not address:

t0  B disconnects from server 2
t1  B reconnects to server 5
t2  manager updates Redis
t3  manager broadcasts invalidation
t4  server 1's cache clears

    Between t0 and t4, server 1 forwards B's messages to server 2 -> nowhere

What should happen to those messages? The design's own store-then-deliver rule from Lesson 4 saves it: the message is already durable in Mnesia before any forwarding is attempted. So a misroute costs a delivery attempt, not the message — it is retried when B's new location is known.

Push invalidation is the right choice and it is not atomic, so the system must be safe during the window. Here it is, because durability precedes routing — which is a good illustration of why that ordering matters beyond crash recovery.

A refinement worth mentioning: rather than broadcasting invalidations to all 200 servers, the manager can invalidate only servers with a cached entry for that user, which requires the manager to track who cached what. That is more state for less traffic, and at 2 billion users with constant churn the traffic is the thing worth saving.

Stateful protocol, stateless servers

Notice the property that makes this survivable: a WebSocket server holds a session, but the message state lives elsewhere — durability in Mnesia, routing in Redis.

So when a server dies, its connections drop, clients reconnect to another server, and the manager updates the mapping. No message is lost, because no server was the authority on any message.

That is the same stateful protocol over stateless servers property that building block identified, and it is what lets a connection tier of 200 machines tolerate losing any of them. The alternative — servers holding undelivered messages in memory — would make every server failure a data-loss event.

Hold the connection on the server and the state somewhere else.

Next: the message service and what happens when nobody is home.