The Finalized Design

Why this matters: this is the diagram to be able to draw from memory, and the comparison that explains why it looks the way it does rather than just what it contains.

Key takeaway

Producers write to brokers; consumers pull from them. The cluster manager handles the broker and topics registry, replication, and authorization. The consumer manager handles verification, retention, delivery mode, and offsets — backed by a key-value store. Storage holds subscriptions and policies.

The whole system

The two designs, side by side

	First design (queues)	Second design (broker)
Copies per message	One per subscriber	One, shared
Publisher cost	O(N) in subscriber count	O(1) — one append
Consumer independence	Full with per-consumer queues; lost with a shared queue and refcount	Full — each has its own offset
Slow consumer impact	Head-of-line blocking in the shared variant	None — cursors don't interfere
Deletion driven by	Consumption	Retention
Metadata volume	One queue per subscriber	One offset per consumer per partition
Ordering	Per queue	Per partition, opt-in via `partition_ID`

The broker architecture resolves the scalability issues of the first design. Partitioning avoids the need for millions of queues and introduces parallelism to prevent bottlenecks.

Checking the requirements

Requirement	How the design meets it
Scalability	Partitions spread write load across brokers and cap consumer parallelism at a tunable number; offsets in a key-value store absorb consumer growth; the cluster manager absorbs topic growth
Availability	Three replicas per partition across different brokers, with the cluster manager electing a new leader on failure
Durability	Append-only local storage plus replication; messages persist for the retention period independent of consumption
Fault tolerance	Per-partition leadership means a broker failure costs one partition's leadership, not a topic; control components are off the data path
Concurrency	Immutable records mean readers never conflict with each other, and writers only ever append — no locking on the read path at all

Conclusion

This chapter compared two pub–sub designs: a queue-based model and a custom storage model optimized for small messages. Publish–subscribe systems decouple producers and consumers, which enables independent scaling and isolates failures between components. This architecture is well suited for large-scale systems that require controlled delivery semantics and high throughput.

What this design does not give you

Worth being able to volunteer, because a strong candidate names the limits of their own design:

No exactly-once delivery. Offset commit and side effect are two operations; ordering them gives at-least-once or at-most-once. Exactly-once requires idempotent handlers or an atomic commit.
No global ordering. Order is per partition. Two messages on different partitions have no defined relationship, ever.
No unbounded history. Retention bounds replay, so it bounds recovery. A bug found after the window cannot be repaired from the log.
No protection from a bad partition key. Keyed writes bypass load balancing, so a skewed key gives a hot partition and the system will not rebalance it for you.
Storage coupled to brokers. Retention capacity is bounded by broker disk, which is why tiered storage into object stores is where these systems are heading.

Key takeaway

Producers append, consumers pull, and both managers stay off the data path. The design beats the queue-based one on every axis because of one substitution — queue to append-only log — which makes copies unnecessary, readers independent, and concurrency a non-issue.

Interview signal by level

Level	What a strong answer sounds like
L4	"Producers write to brokers, consumers read from them, and managers handle the cluster and the consumers."
L5	Separates the planes: "the message path is producer to broker to consumer; both managers are control plane and nothing flows through them, so they can restart without interrupting delivery."
Staff+	Names the substitution and the limits: "every advantage over the queue design comes from replacing the queue with an immutable log — one copy, independent cursors, deletion by retention, and concurrency solved by the data model rather than by locking. What it doesn't give us is exactly-once, global ordering, or history beyond retention — and a skewed partition key will still make a hot partition that nothing rebalances."

Next: the whole thing under interview conditions.