Free preview

High-Level Design and Building Blocks

In one line: the building-block list contains one entry that no requirement asks for, and tracing why it is there uncovers a whole feature.

The high-level picture

The system must support uploading, viewing, and searching media. Additionally, users can follow one another. This requires a robust storage layer to persist media and a retrieval mechanism to fetch data on demand.

Building blocks

BlockRole
Load balancerDistributes requests across available servers
DatabaseStores user metadata and relationships
Blob storageStores media content like photos and videos
Task schedulerManages asynchronous tasks, such as cleaning up expired content
CacheStores frequently accessed content
CDNDelivers content to end-users, reducing latency and server load

The task scheduler is on the list, and no requirement asks for one

Read the five functional requirements again — post, follow, like, search, feed. None of them needs a scheduler.

Yet the block list says: "Task scheduler: manages asynchronous tasks, such as cleaning up expired content."

Expired content appears nowhere in the requirements. It is a forward reference to Lesson 10's Stories feature — photos that vanish after 24 hours — which the design introduces as an addition rather than a requirement.

Two things worth taking from this.

A component in the block list that no requirement justifies is a signal. Either a requirement is missing, or the component is speculative. Here it is the former: the design anticipates ephemerality before specifying it.

Deletion at scale is a scheduling problem, not a storage one. That is the non-obvious part. Expiring a story is not "set a flag" — with hundreds of millions of stories a day, something must find the expired ones and reclaim their blob storage. That is a recurring batch job over a huge keyspace, which is precisely what a distributed task scheduler is for.

Compare it to the newsfeed chapter's tombstones: there, deletion was a read-path filter because the data was small and references were cheap. Here the data is media, and leaving expired videos in blob storage costs real money at 5,430 TB a day. When the deleted object is large, you must actually delete it, and that needs a scheduler.

Cache and CDN are both listed, and they serve different things

Easy to treat as one layer. They are not, and the distinction sharpens as this chapter progresses.

CacheCDN
HoldsMetadata, timelines, hot rowsMedia bytes
LocatedIn the data centreAt the edge, in ISP networks
Sized byWorking setEgress — 50 Tb/s
Miss costsA database readAn origin fetch across the internet

Lesson 3 established that media is 96.7% of the bytes, so the CDN carries almost all the volume while the cache carries almost all the queries.

That split shows up structurally in Lesson 11's finalized design, where reads go to the CDN first and fall back to application servers. It is the only design in this module that puts a CDN ahead of the application tier rather than beside it — and Lesson 3's egress figure is why.

What the block list omits

Three components the chapter needs and does not list.

A search index. Lesson 2's requirement is "search photos and videos based on captions and location," ranked by reach. That is an inverted index over text plus a geo index, and neither the database nor the cache provides it. The Twitter chapter had the same gap.

A transcoding pipeline. 35 million videos a day must be converted into multiple resolutions and bitrates before they can be served adaptively. The YouTube chapter treated this as a first-class subsystem; here it is invisible, even though Lesson 3's storage estimate depends entirely on video.

A counter service. Lesson 11 answers a question about counting "millions of interactions on a celebrity post" with sharded counters — a component that appears in the answer but never in the list, and which Lesson 2 showed is also the search ranking signal.

None of these is fatal to the design, but naming them is the difference between reciting a block list and reading one. The gap between the components a design lists and the components its requirements imply is usually where the interesting questions are.

Key takeaway

The task scheduler is on the block list without a requirement to justify it — a forward reference to Stories — and the reason it is needed is that deletion at scale is a scheduling problem: unlike the newsfeed chapter's cheap tombstone filters, expired media must actually be reclaimed because the objects are large. Cache and CDN are separate layers — the CDN carries almost all the bytes, the cache almost all the queries — which is why this is the only design in the module that puts a CDN ahead of the application tier. And three needed components are absent: a search index, a transcoding pipeline, and a counter service.

Next: the API.

Enjoying the preview?

Create a free account to unlock the rest of this course, the in-browser judge, and live AI mock interviews.

Sign up free to continue