The Deployment Pipeline
In one line: every other chapter designs a system that serves users. This one designs the system that ships every other system — which changes both who the users are and what a failure costs.
What it is
A code deployment system, often called a deployment pipeline or continuous deployment system, automates software distribution across environments like testing, staging, and production. Its primary objective is to release code changes efficiently while minimizing disruption and risk.
This is infrastructure, not a product — and three things follow from that
It is worth noticing how different this problem is from everything else in the module before you start designing.
The users are engineers, and there are few of them. Not 500 million daily actives — a few thousand developers triggering a few thousand deployments a day. So the request rates that dominated every previous chapter are irrelevant here, and the numbers that matter are artifact size and fan-out to machines.
The data is enormous relative to the request count. Three thousand deployments a day is nothing; three thousand 20 GB artifacts replicated to thousands of machines across regions is a lot. The load is measured in bytes moved, not in requests served — which is why the entire interesting engineering here is distribution.
A failure takes down everything else. If the newsfeed is down, the newsfeed is down. If the deployment system pushes a bad artifact to every machine in every region, it takes down the whole company — and it takes down the thing you would use to fix it. That is why the design's non-functional requirements read "every build must definitively report success or failure" rather than a latency target.
When the system you are designing is the one that changes other systems, its blast radius is every system it touches — which makes rollback, staging, and gradual rollout the design's real subject rather than optional extras.
The seven stages
| Stage | What happens | Failure means |
|---|---|---|
| Developers commit to Git; the codebase is tracked | Nothing ships |
| Changes merge to a shared repo; preliminary tests run immediately | A broken merge is caught in minutes |
| Compile, resolve dependencies, produce deployable artifacts (binaries) | No artifact — the pipeline stops |
| Extensive suites detect regressions the change introduced | A regression reaches production |
| Deploy to an environment that mimics production | Production is the first realistic test |
| Download artifacts to target machines; feature flags or gradual rollout | Every machine gets a bad build at once |
| Watch performance; trigger a rollback to the last stable version | The bad build stays up |
CI tests and deployment tests are different tests, and the distinction is about scope
The design poses this as a quiz and the answer is worth internalizing: CI tests validate code changes early; deployment tests validate the full system before release.
The two sit at opposite ends of a real trade-off:
CI tests (stage 2) fast, narrow, run on EVERY commit
-> unit tests, linting, a quick build
-> must finish in minutes or developers stop committing
Deployment tests (4) slow, broad, run before RELEASE
-> integration, regression, end-to-end
-> can take hours; run far less often
The gating constraint on CI is developer feedback latency — a suite that takes an hour means an hour between pushing and knowing, and people batch their commits to avoid it, which defeats continuous integration entirely.
The gating constraint on deployment testing is coverage — this is the last automated gate before real users, so breadth beats speed.
Split a test suite by what its latency budget is for, and you get this shape naturally. It is the same fast-path/slow-path split that separated that building block's edit path from its conversion path, and that building block's suggestion service from its assembler. Two workloads with latency budgets orders of magnitude apart should not share a stage.
The stages that carry the risk
Stages 4, 5, and 7 are the safety net — and the design in this chapter has none of them
Read the seven stages again and notice which ones exist to catch mistakes rather than to move bits:
| Stage | Purpose |
|---|---|
| 1, 2, 3, 6 | Move code forward — commit, merge, compile, distribute |
| 4. Automated testing | Catch regressions before release |
| 5. Staging | Find environment-specific failures before production |
| 7. Monitoring and rollback | Limit the damage when the first two miss something |
Four of the seven stages move code; three exist purely to stop bad code.
Hold onto that, because when this chapter reaches its detailed design, the flow is:
build -> primary blob storage -> regional blob storage -> install on production
Stages 4, 5, and 7 are absent. There is no test gate between build and deploy, no staging environment anywhere in the architecture, and the monitoring service "notifies administrators" rather than triggering the automatic rollback stage 7 promises. Lesson 8 works through this in full.
The reason to flag it here is that the seven-stage model is the right mental checklist for the interview: when you sketch a deployment pipeline, the stages that catch failures are the ones an interviewer is checking for, because they are the ones under time pressure that teams actually skip.
Why "efficiently while minimizing risk" is a genuine tension
Deployment speed and deployment safety pull in opposite directions
The stated objective contains a conflict, and every strategy in the next lesson is a different resolution of it.
FASTER: fewer gates, larger batches, all machines at once
-> ship in minutes, find out about failures from users
SAFER: more gates, smaller batches, gradual rollout with soak time
-> ship in hours, find out about failures from a small blast radius
And there is a counterintuitive second-order effect that experienced teams lean on: deploying more often makes each deployment safer, because a deployment that contains one change has an obvious culprit when it breaks, while one that contains a hundred changes accumulated over a month does not.
That is why the functional requirement specifies builds completing within twenty minutes. It reads like a performance target and it is really a safety mechanism — long build times push teams toward large batches, and large batches are where the risk lives.
Deployment frequency and deployment risk are inversely related, which is the opposite of the intuition. It is the single best thing to say if an interviewer asks why anyone would deploy continuously.
Build once, promote the same bytes
The principle that makes everything downstream verifiable, and it is easy to violate without noticing.
Rebuilding per environment quietly destroys the value of testing. A dependency resolves to a newer patch, a build flag differs, a timestamp changes — and the thing that passed staging is not the thing production runs. The failure is rare and extremely confusing when it happens.
Build once, then promote the identical artifact through every environment. Give it a content-addressed identity — a hash of the bytes — so promotion is a matter of reference rather than reproduction.
That identity is also what makes the security story work: you can sign a hash, and a signature over content-addressed bytes proves the thing running is the thing that was built and reviewed. Immutability is the prerequisite for provenance, which is why these two topics keep appearing together.
Environment differences then belong in configuration injected at deploy time, never baked into the artifact.
Key takeaway
This is the only system in the module whose users are engineers and whose failures take down everything else — so request rates are irrelevant and the numbers that matter are artifact size and fan-out to machines. The seven-stage pipeline splits cleanly: four stages move code, three exist to stop bad code, and it is those three — automated testing, staging, and monitoring-with-rollback — that this chapter's detailed design omits. CI tests and deployment tests differ by latency budget, fast-and-narrow versus slow-and-broad, the same fast-path/slow-path split seen throughout the module. And the objective contains a real tension, resolved by an inverted intuition: deploying more often makes each deployment safer, because small batches have obvious culprits.
Next: the five deployment strategies and what each one costs.