Service Hosts: Collapsing the Tiers
In one line: a logical decomposition does not have to be a physical distribution.
The change
Web and application servers combine onto a single powerful host, eliminating the network I/O and latency of hops between manager, worker, and routing library.
The manager-worker decomposition stays. Managers still generate the page skeleton and fan expensive parts to workers; the routing layer still queues tasks; any worker still takes any task.
What changes is where those processes run. Previously they sat on different machines and every task crossed a network. Now they are on one host and communicate through inter-process channels.
That distinction matters because the parallelism Lesson 3 justified is real and worth keeping — a page still decomposes into concurrent work. What was not worth keeping was the network between producer and consumer.
The three costs, named
Collapsing tiers is not free, and the design accepts each cost for a stated reason.
| Cost | What you give up | Why it's acceptable here |
|---|---|---|
| No independent scaling | You could previously add app servers without adding web servers | The tiers serve the same requests in fixed proportion, so they scale together anyway |
| Shared failure domain | A host failure now takes both manager and worker capacity | Capacity is horizontal across many hosts; losing one loses a slice of both, proportionally |
| Bigger machines | You need one host large enough for both roles | Less flexible than two pools of cheap machines, but the hop was costing more |
That gives the general test: separate tiers when they scale on different signals; collapse them when they scale together. Web and application servers here scale on the same signal — request volume — so the split was buying nothing that justified a per-task round trip.
How the processes talk
Two options are workable: UNIX domain sockets or local TCP connections.
Sockets allow streaming between sender and receiver with flow control — and congestion control, for TCP. That matters because manager and worker produce and consume at different rates, and a stream with backpressure lets the fast side wait rather than dropping work or buffering without bound.
UNIX sockets are the better fit when both ends are guaranteed to be on the same host, since they skip the network stack entirely. Local TCP keeps the option of moving a process off-host later, at the cost of some overhead.
Key takeaway
Keep the manager-worker decomposition and remove the network between them — a logical decomposition does not have to be a physical distribution. Communication moves to UNIX sockets with flow control, so producer and consumer can run at different rates. The design accepts no independent scaling, a shared failure domain, and bigger machines, and it is right to, because the two tiers scale on the same signal. Separate tiers when they scale differently; collapse them when they scale together.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "Put the web and application servers together so there's no network call between them." |
| L5 | Keeps the decomposition: "we collapse them onto one host but keep managers fanning work to workers — it becomes inter-process instead of inter-machine." |
| Staff+ | States the test and pays the costs out loud: "a logical decomposition doesn't have to be a physical distribution — the parallelism was worth keeping, the round trip wasn't. I'd name what we give up: independent scaling, a shared failure domain, and needing bigger machines. That's acceptable because the two tiers scale on the same signal. The rule I'd apply generally is: separate tiers when they scale on different signals, collapse them when they scale together." |
Next: vertical sharding, and why it is not the same fix as YouTube's.