Multi-Tier Architecture and the Long Tail

Why this matters: a flat CDN sends every cache miss to the origin — and with thousands of edges, that is thousands of identical requests for the same object. The hierarchy is what stops the origin from being crushed by its own cache misses.

Key takeaway

Distributing content to all CDN proxy servers simultaneously is resource-intensive and can overload the origin. CDNs use a hierarchical distribution model: edges fetch from parent nodes, which fetch from the origin.

The hierarchy

Edge proxy servers operate within a hierarchy of parent and peer nodes. They fetch content from upstream parents, which in turn obtain it from the origin. Content propagates from origin to proxies along multiple paths in the hierarchy.

The tree structure lets the system scale for increasing users by adding more server nodes to the tree, and it reduces the burden on the origin server for data distribution. A CDN typically has one or two tiers of proxy servers.

Whenever a new proxy server enters the tree, it requests the control core, which maintains information on all proxy servers in the CDN and provides initial content with the configuration data.

The long tail

Research shows that much content has a long-tail distribution: a handful of items are very popular, followed by a long tail of far less popular content.

A multi-layer cache handles long-tail content.

Why the tail needs a different tier

The head is easy: a small number of hot objects fit in RAM at every edge and serve almost all traffic. The tail is the problem — collectively it is a large fraction of requests, but each item is requested rarely, so caching all of it at every edge is enormously wasteful.

The hierarchy resolves this by specializing. Edges hold the head in memory. Parents hold a much larger slice — including the tail — on disk, shared across many edges. A tail request costs one regional hop instead of a global one, and no edge has to store an object it will serve twice a month.

This is why proxy servers keep hot data in RAM and cold data on SSD or disk, and why the tiering exists at all. It is the same head/tail split, expressed twice: once inside a node, once across the tree.

What happens when a node fails

Failure	Recovery
Child proxy fails	DNS directs clients to another child proxy — the routing layer simply stops returning the dead one
Parent proxy fails	Each child knows many upper-layer parent servers and switches to another
Origin server fails	The origin is a set of servers with hot backups, and content is in a replicated store; if one fails, others take the load

Every tier has more than one option upstream, which is the redundancy pattern from the Foundations module applied at each level of the tree — no tier has a single parent whose loss orphans it.

Key takeaway

The hierarchy exists because cache misses aggregate. One tier between edges and origin cuts origin load by the fan-out factor, and gives the long tail somewhere to live that isn't every edge's disk.

Interview signal by level

Level	What a strong answer sounds like
L4	"Edge servers fetch from the origin when they don't have the content."
L5	Adds the tier: "a parent cache layer between edges and origin, so a miss doesn't always reach the source."
Staff+	Quantifies and connects it to the tail: "1,000 edges missing the same object is 1,000 origin requests flat, or 20 with a parent tier — a 50x reduction. It also gives the long tail a home: head in edge RAM, tail on parent disk, so no edge stores what it serves twice a month."

Next: how a client finds the right edge in the first place.