Estimating Storage and Bandwidth

In one line: the server estimate said compute was trivial. Storage and bandwidth are where a service like this actually lives — and where a single assumption about media ratios swings the answer by two orders of magnitude.

Storage requirements

Estimating annual storage for new tweets, from these assumptions:

Daily active users        = 500M
Tweets per user per day   = 3
Tweets containing images  = 10%   (mutually exclusive with video)
Tweets containing video   = 5%
Average image size        = 200 KB
Average video size        = 3 MB
Tweet text and metadata   = 250 bytes

Working it through:

Total tweets/day    = 500M * 3                = 1,500M  (1.5 billion)

Text    = 1.5B * 250 B                        = 375 GB
Images  = 1.5B * 10% * 200 KB = 150M * 200 KB = 30 TB
Videos  = 1.5B * 5%  * 3 MB   = 75M  * 3 MB   = 225 TB

Total daily storage = 0.375 TB + 30 TB + 225 TB ~= 255 TB
Total annual        = 365 * 255 TB              = 93.08 PB

Bandwidth requirements

Bandwidth follows a three-step recipe:

1. Estimate daily incoming data.
2. Estimate daily outgoing data.
3. Divide by seconds in a day to get Gbps.

Incoming

Incoming traffic is what users upload, which is exactly the 255 TB of daily storage computed above:

Incoming = (255 * 10^12 / 86,400) * 8
         ~= 24 Gbps

Outgoing

Assume each user views 50 tweets per day. Applying the same content ratios, 50 tweets contain 2.5 videos and 5 images. With 500M DAU:

Tweets viewed/second = 500M * 50 / 86,400        = ~289K

Text    = 289K * 250 B * 8                        = 0.58 Gbps
Images  = 289K * 10% * 200 KB * 8                 = 46.24 Gbps
Videos  = 289K * 5%  * 3 MB   * 8                 = 346.8 Gbps

Total outgoing = 393.62 Gbps

Total

The service requires 24 Gbps incoming and 393.62 Gbps outgoing (assuming uncompressed uploads), for a total of 417.62 Gbps.

Note the asymmetry: outgoing is roughly 16x incoming, because content is written once and read many times. That read-heavy shape is what makes a CDN the dominant architectural decision here — and it echoes the read/write ratio reasoning from the previous chapter.

These calculations depend heavily on assumptions about traffic mix (text versus media) and the read/write ratio. Change "50 tweets viewed" to 100 and the egress doubles.

The plausibility checks

Now the step that separates an estimate from a number.

Is 93 PB per year plausible?

20 TB disks are readily available. Hosting 93 PB therefore needs about 5,000 disks. Even with three-way replication, that is only 15,000 disks. At a retail price of around 400 USD each, that is roughly 6 million USD — and organizations typically receive substantial volume discounts on hardware at that quantity.

Six million USD of disk for a service with 500 million daily users is entirely reasonable. The number passes.

Is ~400 Gbps plausible?

Many data centers within an organization are connected via high-speed networks such as 1 Tbps. Bandwidth toward the global internet may be higher-cost, but a Twitter-scale organization operates multiple, geographically dispersed data centers whose collective bandwidth easily matches this figure.

Also reasonable. Note that 417 Gbps against the 25 Gbps per-instance network limit from the previous lesson implies at least ~17 instances just to carry the traffic — more than the 8 the CPU estimate suggested. Bandwidth, not compute, sets the floor on fleet size here, which is exactly the kind of finding a BOTEC exists to surface.

Always close the loop

Both checks convert an abstract figure into something physical — disks you could count, links you could buy. That is what makes an estimate defensible rather than merely arithmetical, and it is the habit interviewers are watching for.

If a plausibility check fails, that is equally valuable: it means an assumption is wrong, and you have found it before committing to a design built on it.

The reusable framework

The same shape works for any service:

Interview signal by level

Next: running one of these live, under time pressure.