Standard Numbers Every Designer Should Know

Why this matters: effective planning requires knowing what workloads machines can actually handle. These two tables are the raw material for every estimate in this chapter — and for most estimates you will ever make in an interview.

Key takeaway

Learn the order of magnitude, not the digits. The gap between an L1 cache hit and a cross-country round trip is eight orders of magnitude, and essentially every design decision follows from knowing roughly where an operation sits on that scale.

Important latencies

Latency is the key factor in resource estimation. These are the numbers worth carrying in your head:

Component	Time (nanoseconds)
L1 cache reference	0.9
L2 cache reference	2.8
L3 cache reference	12.9
Main memory reference	100
Compress 1 KB with Snzip	3,000 (3 us)
Read 1 MB sequentially from memory	9,000 (9 us)
Read 1 MB sequentially from SSD	200,000 (200 us)
Round trip within same datacenter	500,000 (500 us)
Read 1 MB sequentially from SSD at ~1 GB/sec	1,000,000 (1 ms)
Read 1 MB sequentially from disk	2,000,000 (2 ms)
Disk seek	4,000,000 (4 ms)
Send packet SF to NYC (round trip)	71,000,000 (71 ms)

Focus on the order-of-magnitude difference between components rather than the exact numbers. For example, IO-bound work (reading 1 MB sequentially from SSD) is roughly two orders of magnitude slower than CPU-bound work (compressing 1 KB of data).

Reading the table as ratios

The absolute numbers are hard to hold. The ratios are not:

L1 cache        1x
Main memory     ~100x slower than L1
1 MB from SSD   ~200,000x
Same-DC round trip  ~500,000x
Cross-country RT    ~79,000,000x

This is the same lesson as Chapter 1's fallacy of zero latency, now with the reference numbers attached. A cross-country round trip costs about 142x a same-datacenter round trip, which is why geographic placement of data is an architectural decision rather than an operational one.

Important rates

Beyond latency, throughput is measured as the queries per second a single server can handle:

Queries	QPS
QPS handled by MySQL	1,000
QPS handled by key-value store	10,000
QPS handled by cache server	100,000 – 1 M

These are approximations. Real performance varies with query type (a point query versus a range query), machine specifications, database design, indexing, and server load.

Why the three tiers differ by 10x each

The progression MySQL → key-value store → cache is not arbitrary; each step removes work:

System	Typical QPS	What it does per request	Why it's faster than the tier below
MySQL	1,000	Parse, plan, optimize, execute, possibly hit disk	Baseline — query planning is real work
Key-value store	10,000	Hash the key, fetch the value	Simpler API (put/get) means no query planning
Cache server	100,000 – 1 M	In-memory read or write	No disk at all; read-dominated workloads go higher still

A key-value store serves an order of magnitude more queries because it has a much simpler API — put and get — versus a relational database that must plan a query before executing it. In-memory caches support read and write operations that are simpler still, and a cache used primarily for reading can serve even more requests per second.

Key takeaway

Treat these like the memory hierarchy: fix one number you trust — say 1,000 QPS for a relational database — and derive the rest from the order-of-magnitude relationships. You then only have to remember one number and two ratios instead of three independent facts.

Numbers worth deriving, not memorizing

A few more that fall out of the tables above and appear constantly:

Quantity	Value	How to remember it
Seconds in a day	86,400 (use 100,000)	Rounding up costs ~15% and makes the division trivial
Seconds in a year	~31.5 million	About pi times 10 million — genuinely close
Bytes to bits	x8	Storage is bytes, network is bits. Forgetting this is an 8x error
1 million/day	~12 per second	1M / 86,400. A useful anchor for scaling any daily figure
1 billion/day	~11,600 per second	1,000x the row above

Interview signal by level

Level	What a strong answer sounds like
L4	Knows memory is faster than disk, without magnitudes.
L5	Uses the numbers: "a same-DC round trip is about 500 microseconds, so 50 sequential calls is 25 ms of pure network."
Staff+	Reasons in ratios and knows the floors: "cross-country is ~71 ms and that's near the speed-of-light limit, so this is a data-placement problem, not a tuning problem. And 2,000 versus 1,000 QPS doesn't change my design — it's the same order of magnitude."

Next: why not all requests cost the same.