Standard Numbers Every Designer Should Know

In one line: effective planning requires knowing what workloads machines can actually handle. These two tables are the raw material for every estimate in this chapter — and for most estimates you will ever make in an interview.

Important latencies

Latency is the key factor in resource estimation. These are the numbers worth carrying in your head:

Focus on the order-of-magnitude difference between components rather than the exact numbers. For example, IO-bound work (reading 1 MB sequentially from SSD) is roughly two orders of magnitude slower than CPU-bound work (compressing 1 KB of data).

Two SSD rows, two different devices

The table lists 1 MB sequential SSD reads twice — at 200 us and at 1 ms — because those describe SSDs of different speeds; the second row states its ~1 GB/sec assumption explicitly. Note also that a disk seek (4 ms) costs more than sequentially reading 1 MB from disk (2 ms). That is not a typo, and it is one of the most useful facts here: on spinning media, finding the data costs more than reading it, which is the entire reason sequential access patterns and batching matter so much.

Reading the table as ratios

The absolute numbers are hard to hold. The ratios are not:

L1 cache        1x
Main memory     ~100x slower than L1
1 MB from SSD   ~200,000x
Same-DC round trip  ~500,000x
Cross-country RT    ~79,000,000x

This is the same lesson as that building block's fallacy of zero latency, now with the reference numbers attached. A cross-country round trip costs about 142x a same-datacenter round trip, which is why geographic placement of data is an architectural decision rather than an operational one.

Important rates

Beyond latency, throughput is measured as the queries per second a single server can handle:

These are approximations. Real performance varies with query type (a point query versus a range query), machine specifications, database design, indexing, and server load.

Why the three tiers differ by 10x each

The progression MySQL → key-value store → cache is not arbitrary; each step removes work:

A key-value store serves an order of magnitude more queries because it has a much simpler API — put and get — versus a relational database that must plan a query before executing it. In-memory caches support read and write operations that are simpler still, and a cache used primarily for reading can serve even more requests per second.

Numbers worth deriving, not memorizing

Powers of two, because storage arithmetic runs on them

2^10 = 1,024        ~= 1 thousand   -> KB
2^20 = 1,048,576    ~= 1 million    -> MB
2^30                ~= 1 billion    -> GB
2^40                ~= 1 trillion   -> TB
2^50                                -> PB

The useful simplification is that 2^10 is close enough to 1,000 that you can do the whole calculation in powers of ten and be within 2.4% per step — which is far inside the precision an estimate deserves. Nobody in an interview wants 1,073,741,824; they want "about a billion".

Two anchors make the ladder concrete: one character is a byte, so a thousand-character message is a KB; and a minute of video is roughly 50 MB at ordinary quality, which is the number most media estimates start from.

A few more that fall out of the tables above and appear constantly:

Going deeper (Staff+)

The final row of the latency table is a physics constraint, not an engineering one. Light travels about 300,000 km/sec in vacuum and roughly 200,000 km/sec in fiber. San Francisco to New York is ~4,000 km, so a round trip is ~8,000 km, giving a floor near 40 ms — and 71 ms is what you actually get after routing, switching, and a non-straight path.

Saying "that's within about 2x of the speed-of-light floor, so no amount of engineering will fix it — we have to move the data closer" is a genuinely strong interview moment. It reframes a latency problem as a placement problem.

Interview signal by level

Next: why not all requests cost the same.