Timeline Generation and Fan-Out

In one line: this is the problem people mean when they say "design Twitter," and this chapter does not contain it. It is worth having properly, because the answer is a genuinely elegant hybrid rather than a choice between two options.

The problem

Lesson 2 separated the two timelines. The user timeline is a range scan. The home timeline is the hard one:

I follow 500 accounts.
Show me their tweets, newest first, 20 at a time, instantly.

Done naively, every refresh is a 500-way merge across 500 authors' tweet lists. At 289,000 timeline reads per second, that is 144 million lookups per second.

There are two ways out, and they sit at opposite ends of a single trade.

Fan-out on write — push

At post time, copy the tweet ID into every follower's precomputed timeline. A read is then a single lookup of an already-assembled list.

Fan-out on write dies on celebrities, and the arithmetic is brutal

For an account with 500 followers, posting is 500 writes. Fine — they happen asynchronously, and Lesson 2's requirement explicitly permits it.

For an account with 50 million followers, posting one tweet is 50 million writes.

50,000,000 timeline insertions from ONE user action

Three things break at once:

Latency. Even at a million writes per second of fan-out capacity, that is 50 seconds before the last follower sees it. During breaking news, "eventually" is not good enough.

Thundering herd. Several large accounts posting at once produces a write burst that no steady-state capacity plan covers.

Waste. Most of those 50 million followers are inactive. You have just written to millions of timelines nobody will read.

That last point is the real indictment. Fan-out on write pays the cost per follower, but the value is per active reader, and for large accounts those numbers diverge by orders of magnitude.

This is the heavy-hitter problem from Lesson 10, appearing in the write path rather than the counter path. Same root cause: a power-law distribution where the extreme case is thousands of times the median.

Fan-out on read — pull

At post time, write once. At read time, look up who I follow, fetch their recent tweets, and merge.

Pull is correct and too slow, because the system is read-heavy

Pull inverts every property of push. Posting is trivial; reading is expensive.

And that is the wrong way round for this system. Lesson 3 established 289,000 reads per second against 17,361 writes. Putting the expensive work on the path that runs seventeen times more often is exactly backwards.

Worse, the cost is paid repeatedly. A user who refreshes five times re-merges the same 500 authors five times. Fan-out on write pays once and serves many reads.

Push amortizes; pull recomputes. In a read-heavy system, amortizing is what you want — which is why the default is push and pull is the exception.

The hybrid — and why it is not a compromise

Push for most accounts, pull for celebrities, merge at read time

The real answer uses both, chosen per author by follower count:

Author has < threshold followers  ->  PUSH   (fan out at write time)
Author has > threshold followers  ->  PULL   (fetch at read time)

Home timeline = precomputed list  +  merge in celebrity tweets on read

This is not splitting the difference. It applies each strategy exactly where its cost is low:

The asymmetry that makes it work: there are very few celebrities. If I follow 500 accounts, perhaps 5 are large. So the read-time merge is 5 lookups, not 500 — cheap enough to sit on the read path.

Meanwhile the 495 ordinary accounts were pushed, so their tweets are already in my precomputed list.

Each strategy is applied to the population where its dominant cost is smallest. The hybrid works because the follower distribution is a power law — and a power law means the extreme cases are both very expensive and very few.

Note that the read-time merge step is already required for other reasons. Lesson 4 showed that ads must be spliced in at read time, deleted tweets filtered via tombstones, and protected-account visibility re-checked. So celebrity pull is not a new stage — it is extra work in a stage that has to exist anyway.

The threshold is a real tuning decision with pressure from both sides

Where do you draw the line? Both directions cost something.

Threshold too low — too many accounts classified as celebrities, so the read-time merge grows and every timeline read gets slower.

Threshold too high — some very large accounts still fan out, producing the write bursts push was meant to avoid.

Better than a fixed follower count is a cost-based rule: push when followers × probability-of-being-read is below the cost of a read-time merge. That naturally captures the real insight — an account with a million inactive followers should be pulled even if a million-follower account with engaged followers should be pushed.

A refinement worth mentioning: fan out only to active users. Someone who has not opened the app in six months does not need a materialized timeline; rebuild theirs on demand when they return. At Twitter's scale, inactive users are the majority, so this alone removes most of the fan-out cost.

What the timeline actually stores: IDs, not tweets

An important detail that changes the storage arithmetic.

Timelines store tweet IDs, not tweet content:

BAD:   timeline = [full tweet, full tweet, ...]     ~250 B each
GOOD:  timeline = [id, id, id, ...]                  8 B each

Three reasons this matters:

Storage. At 8 bytes per entry, a 1,000-entry timeline is 8 KB. Storing full tweets would be 250 KB per user, times hundreds of millions of users.

Edits and deletes. With IDs, a deleted or edited tweet is fixed in one place. With copies, you would chase millions of duplicates — Lesson 2's tombstone problem, made far worse.

Counters. Like and retweet counts change constantly. Copying them into timelines would mean rewriting every copy; storing IDs means the counts are fetched fresh from Lesson 10's sharded counters at read time.

So the read path is: fetch the ID list, then hydrate — batch-fetch the tweet bodies from cache, and the counts from the counter service.

Fan out references, not values. The rule generalizes: in any denormalized structure, copy the key and resolve the body, unless the body is immutable and small.

Follow and unfollow are the operations nobody plans for

One consequence of precomputation that is easy to miss: changing who you follow invalidates your timeline.

Follow someone and their recent tweets should appear in your timeline — but fan-out already happened, so they are not there. Either backfill their recent tweets into your list, or accept that you only see their future posts.

Unfollow and their tweets should disappear from a list that already contains them. Either scrub your timeline, or filter on read.

Neither is free, and the usual answer is the cheaper one: backfill on follow (a bounded job — fetch their last N tweets and merge) and filter on read for unfollow, since the read path already has a filtering stage.

It is the same conclusion as tombstones for deletion. In a precomputed system, removal is a read-path concern and addition is a write-path one, because adding is bounded and removing is not.

Interview signal by level

Next: the storage systems Twitter actually built.