Trade-offs: CDN Strategy and Duplicate Videos
In one line: the CDN decision is one of the few genuinely financial trade-offs in system design, and duplicate detection is a rare storage optimization with a legal dimension.
Public versus private CDN
Public CDNs suit low-traffic regions for three reasons. They avoid the capital cost of building infrastructure. They deploy in minutes, which matters when traffic goes viral. And they are cost-effective where user density could never sustain a private node.
As traffic scales, that flips.
The economics are the design. Public CDNs charge per gigabyte delivered — a variable cost that scales linearly with traffic forever. A private CDN is capital expenditure: expensive up front, then cheap per byte.
Cost
| public CDN (linear in traffic)
| /
| /
| / ___________ private CDN (high fixed, low marginal)
| / _/
| /_/
| /
+--------------------------------- traffic
^ crossover
Below the crossover, renting wins. Above it, owning wins — and at 12 Tbps, YouTube is very far above it in its major markets.
Three things make this more nuanced than a single threshold.
It is per-region, not global. A region with a handful of viewers never reaches the crossover no matter how large the company is. So the answer is a hybrid: own where density justifies it, rent everywhere else.
Viral traffic is unpredictable, and public CDNs absorb it instantly. Building capacity takes months; renting takes minutes. Public CDNs therefore also function as elastic overflow.
Owning buys optimization, not just cost. A private CDN can implement its own placement policy, custom eviction, and ISP peering arrangements that a rented service will not.
The transferable framing: rent while the cost is variable and the volume is uncertain; build when the volume is predictable and large enough to amortize the fixed cost.
What duplicates cost
Assume 50 of every 500 uploaded hours are duplicates, at 6 MB per minute:
Wasted per minute = (50 hours x 60 min/hour) x 6 MB/min
= 3,000 x 6 MB
= 18,000 MB
= 18 GB per minute
Wasted per year = 18 GB/min x (60 x 24 x 365) minutes
= 18 GB x 525,600
= 9,460,800 GB
~ 9.5 petabytes per year
A 10% duplication rate is conservative for a platform with free uploads and no barrier to re-posting. And 9.5 PB is one compressed quality — apply the multipliers from Lesson 2, five renditions plus raw originals, and duplicates cost closer to 50–100 PB annually.
This is pure waste. Not a long tail of unpopular content that might someday be watched, but copies of things already stored. That makes deduplication one of the highest-return optimizations available, because the saving is total rather than marginal.
It also explains the ordering in Lesson 3: dedupe runs before transcoding, so a duplicate costs neither the storage nor the compute across five resolutions.
Duplicates are a copyright problem first
Storage is measurable and payable — 9.5 PB is expensive, but it is an invoice. Copyright infringement is legal exposure, and at platform scale that is existential rather than expensive.
That reframing changes the requirements:
- Detection must survive modification. A re-upload with a watermark, slight crop, changed resolution, or shifted audio is still infringing, and byte-level hashing catches none of it.
- It must run on every upload, not opportunistically the way a storage optimization would.
- False negatives cost more than false positives, which is the reverse of most systems.
The escalation to block matching and then machine learning follows the difficulty of the modification. Worth being honest in an interview that this is an adversarial problem — people re-uploading copyrighted content actively try to defeat detection, so the system is never finished.
Key takeaway
Public versus private CDN is a build-versus-buy crossover, decided per region — rent for elasticity and low density, own where volume amortizes the fixed cost and buys optimization control. Duplicates cost 9.5 PB/year at one quality and several times that in practice, which is why dedupe happens before transcoding. The real driver is copyright exposure, which demands perceptual matching rather than byte-level hashing.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "Use a CDN for delivery, and detect duplicate uploads to save storage." |
| L5 | Frames the CDN economically: "public CDN while traffic is low or unpredictable, private once volume justifies the capital cost — and it's a per-region decision, so realistically a hybrid." |
| Staff+ | Names the crossover and reframes duplicates: "public CDNs are variable cost scaling linearly forever; private is fixed cost with cheap marginal delivery, so there's a crossover per region — and public also serves as elastic overflow for viral traffic we can't provision for in months. On duplicates, 10% duplication is about 9.5 PB a year at one quality and several times that with renditions and raw — but the real driver is copyright exposure, not storage. That means perceptual matching with LSH rather than cryptographic hashing, because a re-encode changes every byte, and it's adversarial since people actively try to defeat it." |
Next: the database tier that breaks first.