Encode: Per-Shot Encoding

In one line: this is the first of three lessons on making delivery actually work, and it is the one with the largest compounding effect — every byte saved here is saved again in storage, in edge caching, and in every stream.

The three phases

Per-shot encoding

Each segment will be encoded using multiple schemes, producing multiple files, or chunks. For any video with dynamic colors and high depth, we'll encode it differently from a video with fewer colors. A not-so-dynamic segment will be compressed more to save additional storage space.

Whole-video encoding              Per-shot encoding
-------------------              -----------------
one scheme for everything        segment 1: static talking head  -> compress hard
                                 segment 2: fast action, confetti -> compress lightly
                                 segment 3: slow pan              -> compress hard
                                 segment 4: rapid cuts            -> compress lightly

sized for the WORST segment      each segment sized for ITS OWN content

Whole-video encoding is priced by its most demanding moment — that is the waste

This is the insight, and it is worth stating plainly.

If you encode an entire video with one bitrate, that bitrate must be high enough for the hardest segment. A ten-minute interview with three seconds of confetti gets encoded as though the whole thing were confetti — because dropping the bitrate to suit the talking head would make those three seconds look terrible.

So every easy segment is over-provisioned to protect the hard ones. In typical content, easy segments are the overwhelming majority.

Per-shot encoding removes that coupling: each segment is encoded for its own complexity. Static scenes compress enormously; complex ones get the bits they need.

The generalizable principle: when a single parameter must satisfy a heterogeneous workload, it gets set by the worst case and wastes resources on everything else. The fix is always to make the unit smaller so the parameter can vary — the same reasoning as per-shard eviction policies and per-tenant rate limits.

The savings compound through three separate systems

A byte not encoded is a byte not paid for four times over:

Storage. Lesson 2 put this at 900 GB/min across five qualities. A 20% encoding improvement is 180 GB/min saved — permanently, and multiplied by every rendition.

Edge cache capacity. because the per-shot encoding scheme reduces storage overhead, a larger portion of the catalogue can be cached at edge locations, increasing cache hit rates and reducing origin load. Smaller files mean more titles fit in the same cache, which improves hit rate non-linearly — a cache holding 20% more of a long-tail catalogue catches meaningfully more requests.

Delivery bandwidth. At 12 Tbps, a 20% reduction is 2.4 Tbps — recurring, forever.

User experience. Smaller chunks mean faster start-up and fewer rebuffering events on constrained connections.

That is why encoding earns disproportionate engineering investment at this scale. Most optimizations improve one thing; this one improves storage cost, cache hit rate, egress cost, and quality of experience simultaneously.

The processing pipeline is a DAG

Encoding is not one job. It is a graph of jobs, most of which are independent, which is what lets a ten-minute video finish in wall-clock minutes rather than in real time.

Split first, then fan out. Every segment at every quality is an independent unit of work, so wall-clock time is set by the size of the worker pool rather than by the length of the video.

The split has to be GOP-aligned. A compressed video only carries full frames at keyframes; everything between is a delta against them. Cut mid-GOP and the segment cannot be decoded on its own, and the client cannot switch quality at that boundary. So segment boundaries are keyframe boundaries, forced identical across every rendition — which is what makes the renditions interchangeable at playback time.

A queue sits in front of the pipeline. Upload arrival is bursty and transcoding capacity is not elastic on the same timescale, so the queue absorbs the burst, gives per-task retry, and lets workers scale on queue depth. It also means a transcoding outage delays videos rather than failing uploads.

Audio and subtitles

We can also encode audio in various formats to optimally allow streaming for various clients like TVs, mobile phones, and desktop machines. Specifically, for services like Netflix, audio encoding is more useful because audios are offered in various languages.

Separating audio from video is what makes multi-language economical

If audio were muxed into the video file, offering a film in ten languages would mean storing ten complete copies of the video — with identical pixels.

Keeping them as separate streams means one set of video renditions plus ten small audio tracks, assembled at playback. The video is the expensive part; the audio is not.

This matters more for Netflix than YouTube — a service with professionally localized catalogues has many audio tracks per title, while most YouTube content has one.

It also explains the combinatorics the design has to manage. A title is not one file; it is:

segments x video qualities x codecs  +  audio languages x formats  +  subtitle languages

Which is exactly the complexity Lesson 4 flagged when it noted the schema's video_URI should really be a one-to-many rendition mapping.

The cost of per-shot encoding is combinatorial explosion in file count

The design presents this as a clear win, and it is — but the bill is worth naming.

Whole-video encoding at five qualities produces five files. Per-shot encoding of a ten-minute video split into, say, four-second segments produces 150 segments × 5 qualities = 750 files for the video alone, before audio and subtitles.

That means:

• Vastly more metadata — every chunk needs a location, which is the mapping Lesson 4 said was omitted for simplicity.
• More encoding jobs, though these parallelize beautifully (and Lesson 12 notes segmentation enables parallel transcoding, which is a benefit rather than a cost).
• More objects in blob storage — fine, given object storage's flat namespace has no directory bottleneck, but the object count is real.

So per-shot encoding trades file count and metadata complexity for bytes. At 12 Tbps, bytes are the expensive resource and metadata is cheap — which makes it the right trade here, and would not be for a system with few viewers and tight metadata budgets.

Interview signal by level

Next: getting the bytes close to users.