The Five Deployment Strategies

In one line: the strategy you pick determines how much infrastructure you need, how fast rollback is, and how many users see a bad build. It is the highest-leverage decision in the chapter, and the design later picks the weakest option.

Basic deployment

We release new code directly to the production environment. While simple, this approach can be risky, as it provides no safety net or rollback mechanism in case issues arise. It's generally unsuitable for large-scale or critical systems.

Basic deployment's real problem is that failure is discovered by users

Every server gets the new version simultaneously, so:

Rollback means another full deployment. There is no old version standing by — recovering requires re-running the whole pipeline with the previous artifact, which takes as long as the deployment did. If the deployment took twenty minutes, your outage is at least twenty minutes.

The blast radius is 100% of users, immediately. There is no window in which a small fraction is affected and monitoring can react.

The failure signal comes from users, not from the system. With no gradual rollout there is nothing to compare against — no cohort still on the old version showing you what "normal" looks like.

That last point is the one people miss. Gradual strategies are not just about limiting damage; they give you a control group. A canary at 5% is measurable precisely because 95% is still running the old version, so a spike in errors is unambiguous. Deploy to everything at once and a 3% error rate might be the new build or might be Tuesday.

A deployment strategy that updates everything at once cannot distinguish a bad release from normal variation. That is why basic deployment is unsuitable at scale, and it is what this chapter's design nonetheless adopts — Lesson 8 covers the consequences.

Multi-service deployment

Multiple interconnected services are updated simultaneously as part of a coordinated release. This ensures consistency across related services and prevents compatibility issues caused by mismatched versions. While it supports cross-service changes, coordinating the release of multiple services can be complex.

This solves a problem that is usually better avoided than coordinated

The problem is genuine: if service A's new version calls an endpoint that only service B's new version provides, deploying A first breaks it and deploying B first is fine — but you have to know that, for every pair, every time.

The strategy's answer is to deploy them together. That works and it has costs that compound:

You cannot roll back one service independently. A bug in A means rolling back both, including B's unrelated fixes.

The deployment is atomic in intent and not in practice. There is no distributed transaction across deployments — A's machines and B's machines update over a window of seconds or minutes, and during that window mismatched versions are talking to each other. Multi-service deployment shrinks the window rather than eliminating it.

It reintroduces the coupling microservices exist to remove. Two services that must ship together are, operationally, one service with extra network hops.

The alternative that production systems actually use is backward-compatible change in two phases:

Phase 1: Deploy B with the new endpoint, KEEPING the old one. B now serves both.
Phase 2: Deploy A to use the new endpoint. Any order works; any rollback works.
Phase 3: Later, remove the old endpoint from B.

Each deployment is independently safe and independently reversible. It costs one extra release cycle and a period of carrying both interfaces.

Prefer making changes compatible over making deployments simultaneous — the first scales to hundreds of services, the second does not. This is the deployment-time form of the schema-migration discipline: expand, migrate, contract.

Rolling deployment

Gradually updates a subset of servers while the others continue serving the old version. This minimizes downtime and distributes load, but is slower than other strategies.

Rolling is the default for a reason: it needs no extra infrastructure

Blue-green needs a duplicate environment. Canary needs traffic-splitting infrastructure. Rolling needs neither — you already have N servers behind a load balancer, and you update them a few at a time.

That is why it is the default in Kubernetes, in every cloud autoscaling group, and in most organizations.

The costs are two, and the second is the one that bites:

It is slow. Batch size trades speed against capacity — update 10% at a time and the deployment takes ten rounds, but you never lose more than 10% of capacity.

Both versions run simultaneously, for the whole deployment. This is unavoidable and it is a real constraint on what you are allowed to ship:

During a rolling deployment, v1 and v2 are BOTH live and BOTH serving traffic.
  -> they read and write the SAME database
  -> they consume the SAME message queues
  -> a user's two requests may hit different versions

So every rolling deployment requires that v1 and v2 be mutually compatible — the same discipline the multi-service section arrived at, now forced by the strategy rather than chosen. A schema change that v1 cannot read is a broken deployment, not a broken database.

Rolling deployment is cheap in infrastructure and expensive in compatibility discipline. Most teams discover the second half the first time a migration goes out mid-rollout.

Blue-green deployment

Maintains two identical environments (blue and green). One serves live traffic while the other updates. This offers instant rollout and rollback but requires duplicate infrastructure.

The property worth paying for is that rollback is a load balancer change

Everything else about blue-green follows from one fact: the old version is still running, fully warmed, serving nothing.

Rolling rollback:     re-deploy the old artifact to every machine   -> minutes
Blue-green rollback:  point the load balancer back at blue          -> seconds

That is the difference between an incident lasting a coffee break and one lasting a sentence. And it means rollback is not a deployment — it does not go through the pipeline, does not need the build system to be healthy, and does not depend on the artifact still existing in blob storage.

Two more properties fall out for free:

Only one version serves traffic at a time. Unlike rolling, there is no window of mixed versions on the request path, which removes an entire class of compatibility bug.

The new environment can be fully validated before receiving traffic — smoke tests, warm caches, verified health checks — while zero users are exposed.

The cost is the obvious one: double the infrastructure, which for a large fleet is a large bill. Note it is not double forever — the standby can be scaled down between deployments, at the price of a cold environment when you need it.

This is the same instinct as that building block's atomic pointer swap for publishing a rebuilt trie, and that building block's primary-secondary swap: build the new thing beside the old one and switch a pointer, keeping the old one live so the switch is reversible. Blue-green is that pattern applied to whole environments.

Canary deployment

Releases code to a small subset of servers or users before rolling it out to the rest. This allows early bug detection with limited impact but requires complex traffic routing and monitoring.

Canary is the only strategy that gives you a measurement, and that is the whole point

Every other strategy limits damage. Canary produces evidence.

Because the old version keeps serving the majority, you have a control group running under identical real conditions — same time of day, same traffic mix, same downstream dependencies. So a difference in error rate between the two cohorts is attributable to the release and to nothing else.

Canary  5%: errors 0.9%
Control 95%: errors 0.3%
  -> the release is the cause. Nothing else differs.

That is why the design's cost — "requires complex traffic routing and monitoring" — is understated in one direction and overstated in the other. Weighted routing is a checkbox on any modern load balancer or service mesh. The hard part is the monitoring: you need per-cohort metrics, a baseline, a decision rule, and enough traffic through the canary for the comparison to be statistically meaningful. A 1% canary on a low-traffic service takes hours to say anything.

Canary trades deployment time for confidence, and it only works if you can measure the two cohorts separately. Teams that add canary routing without per-cohort telemetry get the slowdown and none of the benefit.

Choosing

The design asks how to choose for a system that cannot afford downtime — here is the answer

"How would you decide which deployment strategy to use for a system that cannot afford any downtime?"

Basic is eliminated immediately — rollback is a full redeploy, so a bad release is downtime.

Of the remaining three, the honest answer is canary and blue-green together, not either alone, because they solve different halves:

Canary     answers "is this release bad?"          -> detection
Blue-green answers "how fast can I undo it?"       -> recovery

A canary tells you something is wrong while only 5% of users are affected. Blue-green lets you undo it in seconds rather than in a redeploy. Zero downtime needs both a small blast radius and a fast reversal, and the two properties come from different mechanisms.

In practice this is what large systems run: a canary stage that gates promotion, then a rolling or blue-green rollout to the rest, with automated rollback wired to the canary's metrics.

If asked to pick one, pick canary and say why — limiting the blast radius is worth more than accelerating the fix, because a fast rollback of a release that already hit everyone still means everyone saw the failure.

One caveat to state: none of these help if the change is not reversible. A deployment that runs a destructive schema migration or publishes a message consumers have already acted on cannot be rolled back by any strategy. Deployment strategies protect against bad code, not against irreversible side effects — those need the expand-migrate-contract discipline instead.

A sixth option: shadow traffic

Shadow (or dark) deployment sends a copy of live traffic to the new version while the old one continues serving. The new version's responses are thrown away, so users cannot be affected at all.

It answers a question canaries cannot: does this behave correctly on real production traffic — the awkward shapes, the unexpected encodings, the long tail of inputs no test suite contains — without any blast radius.

Two limits worth naming. It cannot validate anything with side effects, since duplicated writes would actually happen, so shadowing usually requires stubbing or sandboxing the write path. And it doubles the load for whatever fraction you mirror, which is a real capacity cost.

Best used for exactly the case where correctness is hard to assert offline: a rewritten service, a new model, a changed query engine.

Next: requirements and the estimation that disagrees with itself.