Running It Across Data Centres

In one line: the chapter's solution to geographic distribution is genuinely clever and it introduces a failure mode the chapter names honestly and does not fix.

The problem

How will we maintain a unique mapping if redirection requests can go to different data centers that are geographically apart? Does our design assume our DB is consistent geographically?

Why eventual consistency is acceptable here — and the reason is specific

The chapter's justification for relaxing global consistency is unusually well-observed:

Because there is typically a delay between URL creation and the first access request, eventual consistency across geographically distributed databases is acceptable.

That is a genuine property of this workload, not a general hand-wave. Think about how a short URL is used:

t=0     create the short link
t=?     paste it into an email, a tweet, a slide, a poster
t=?+    someone eventually clicks it

The gap between creation and first click is at minimum the time it takes a human to share it — seconds at the very best, usually minutes or longer. Cross-region replication runs in tens or hundreds of milliseconds.

The workload has a built-in grace period, and it is far longer than replication lag.

That is a much stronger argument than the usual "staleness is tolerable here." Compare Uber, where a stale driver location matters immediately, or a payment, where staleness is unacceptable at any duration. Here the system is structurally protected by how the product is used.

Worth having ready as a general form: when the write and the first read are separated by a human action, replication lag is invisible.

The solution: put the region in the URL

A simple way of achieving this is to introduce a unique character in the short URL. This special character acts as an indicator for the exact data center.

Example: service.com/x/short123/ where x indicates the data center containing this record.

The identifier carries its own routing information — elegant, and it costs a character

This is a genuinely nice idea and worth recognizing as a pattern.

Normally, finding which shard holds a key means consulting a directory — that building block's key-value store mapping segments to servers, or ZooKeeper holding shard topology. That directory is a lookup on the critical path and a component that must be highly available.

Here the routing information is embedded in the key itself. Any node can determine the owning data centre by reading one character, with no lookup at all.

DIRECTORY-BASED:  key -> consult directory -> shard -> query
SELF-ROUTING:     key -> READ ONE CHARACTER -> shard -> query

The costs are real and worth naming:

One character of the identifier is spent on routing. With base 58, a single character distinguishes up to 58 data centres — generous — but it makes every URL one character longer, which Lesson 10 showed is not free given the readability requirement.

The mapping is permanent. A URL created in data centre x says so forever. You cannot migrate its record to another region without invalidating a link that is already printed on things. Embedding routing in an identifier means the routing decision is as immutable as the identifier.

Rebalancing becomes impossible. With consistent hashing you can move key ranges between nodes. Here, key x/short123 must live in x for as long as the link exists.

Self-routing identifiers trade flexibility for the elimination of a lookup, and that trade is right when lookups are hot and topology is stable — which describes this system exactly.

And here is the single point of failure the chapter admits

If a short URL request is routed to the wrong data center, the system redirects it to the correct one. However, if the data center responsible for that short URL is unreachable and the URL is not already cached, the request cannot be resolved.

That is honest, and it is a serious consequence.

The availability requirement in Lesson 2 was unusually strong — "the service's domain is part of every generated URL," so downtime breaks links that exist in artifacts you do not control. And this design says: if one region goes down, every URL created there becomes unresolvable.

Not degraded. Not slow. Broken, for the duration.

The chapter's mitigations are partial:

Caching helps only for URLs already accessed. A link created in the failed region but not yet clicked has nothing cached anywhere — and Lesson 12's own grace-period argument says the gap between creation and first click is exactly when a link is most likely to be uncached.

GSLB routes around failures, but it can only route to a region that has the data, and by construction only one does.

The fix the design does not take: replicate each region's records to at least one other region. The routing character then becomes a hint — try the home region first, fall back to a replica — rather than an authority.

That preserves the lookup-free fast path and removes the single point of failure, at the cost of the replication the design was trying to avoid.

A self-routing identifier that names exactly one owner makes that owner a single point of failure for every key it holds. The fix is to make the embedded location a preference rather than a constraint.

Cache misses in a data-centre-local cache

Scenario: an unknown redirection request arrives at a data centre. As the local cache would not have that entry, it fetches the record from the globally consistent database and places it in the local cache for future use.

Straightforward read-through caching, and the chapter's choice of a data-centre-specific cache rather than a global one is right for the reason it gives — minimizing latency.

Two things worth adding.

The cache warms per region independently. A link that goes viral in Europe warms European caches and leaves Asian ones cold. That is correct behaviour, since traffic for a given link is usually geographically concentrated, but it means the effective hit rate is lower than a single global cache would achieve.

Update and delete become harder. Lesson 2's update requirement means a mapping can change. With N independent regional caches, an update must invalidate all of them — and the design has no invalidation mechanism. Short TTLs are the usual pragmatic answer, at the cost of a lower hit rate.

Independent regional caches trade hit rate and invalidation complexity for latency, which is the right trade when reads are hot and writes are rare — 7,600 against 76, per Lesson 3.

GSLB does two different jobs here

The chapter invokes global server load balancing twice — for latency ("distribute traffic across global servers") and for failure ("especially during regional failures").

Worth separating, because they use different signals:

And note the tension with the previous callout: GSLB routing a request to the nearest region is fine, because the region will forward it to the owner if needed. But GSLB routing around a failed region cannot help if the failed region is the only one holding the data.

Load balancing can route around a failed server; it cannot route around missing data. That distinction is what makes the replication gap matter.

Partitioning the ID space rather than sharing a counter is the general move: make the regions independent by construction instead of coordinating them at runtime.

Next: checking the design against its requirements.