Peak Load: The Assumption That Changes Everything

Why this matters: the previous lesson produced 2 servers from a uniform-traffic assumption. This lesson produces 157,000 from the same inputs. Nothing changed except how traffic is assumed to arrive — which makes the distribution assumption the single most consequential number in any estimate.

The worst case: everything at once

To estimate peak capacity, assume a worst case in which all daily requests arrive simultaneously. More accurate estimates require actual request and response distributions, which are typically available only at the prototyping level; alternatively you might assume a statistical model such as a Poisson distribution.

To keep it simple, use DAU as a proxy for peak load in a specific second — treating the total daily volume as though it landed in one instant:

Servers needed at peak = Number of requests/second / RPS of server
                       = 10 billion / 64,000
                       = 157K servers

If all workloads arrive simultaneously and each server handles 64,000 RPS, we need approximately 157,000 servers.

That is almost certainly unfeasible. Which leaves two options: make each server handle more, or make the peak assumption less pessimistic.

Option 1: Improve the RPS of a server

If the peak load assumption holds, you must increase server capacity. Suppose you are limited to 100,000 servers:

Number of requests/second / Max servers we can employ
  = 10 billion / 100,000
  = 100,000 requests per second per server

Going from 64,000 to 100,000 RPS per server requires significant engineering optimization. It is possible — organizations do it — but it is an R&D program, not a configuration change:

These examples show improving RPS is achievable — but it demands focused R&D and financial investment. It is rarely the first lever to reach for in an interview.

Option 2: Improve the peak load assumption

The more productive move is usually to challenge the assumption itself, because "all daily traffic in one second" is not a real traffic pattern — it is a pathological bound.

Apply the Pareto principle (the 80/20 rule): assume 80% of traffic occurs within 20% of the time. Twenty percent of a day is a 4.8-hour window:

                     (0.8 * 10 billion) / (4.8 * 60 * 60 seconds)
Servers needed  =    -------------------------------------------
                                    64,000

                =    462,963 / 64,000

                ~=   8 servers

Eight servers. From the same 500M DAU and the same 64,000 RPS machine.

Note that the Pareto calculation carries its own assumption: requests are distributed equally within that 4.8-hour window. Concurrent versus spread-out requests significantly impact resource needs, so systems built on these assumptions require monitoring to ensure the limits are not violated in practice.

And when load does exceed predictions, you need runtime defenses rather than more arithmetic: load shedding, circuit breakers, and throttling — the Chapter 1 mechanisms, now with a capacity number explaining why they exist.

When the flash crowd actually arrives

Consider a service hosting the dynamic, personalized website of a large news organization. An unexpected major event occurs, and flash crowds arrive to find updates — plausibly a situation where all the DAUs really do come at once. That breaks every load assumption above.

You cannot provision for it. You degrade for it:

Each request and response becomes fast, users get their data from CDN edge nodes near them, and the origin stops being the bottleneck.

Choosing your peak factor

In practice you pick a multiplier over average and defend it:

Interview signal by level

Next: what all this actually costs.