Use Cases and Requirements

Why this matters: the requirements list contains one item that no messaging queue chapter had — a retention policy — and it is the requirement that forces the whole storage design.

Key takeaway

Publish–subscribe messaging enables asynchronous communication between producers and consumers. This decoupling improves scalability, fault isolation, and independent service evolution.

Use cases

Use case	What it means
Improved performance	Push-based distribution eliminates the need for recipients to poll for updates, which reduces latency and improves response times
Handling ingestion	Efficiently handles log ingestion. It buffers large data streams — such as user interactions — before delivering them to analytical systems. Meta's Scribe uses this pattern to route logs to specific consumers or archive them, managing enormous data volumes
Real-time monitoring	Raw or processed messages can feed multiple monitoring applications simultaneously for real-time system oversight
Replicating data	Distributes state changes asynchronously — a leader node pushing updates to followers or distributed caches. Also synchronizes state across multiple client views (web and mobile) in real-time apps like WhatsApp

Using pub-sub in a chat application

"You're designing a real-time chat application with channels for different discussions. Map it onto pub-sub."

The mapping is direct, and it is a common interview warm-up:

Pub-sub concept	Chat application
Topic	A channel — one topic per channel, so `#engineering` and `#random` are separate topics
Publisher	Any member sending a message to that channel
Subscriber	Every member who has joined the channel — including their web and mobile clients as separate consumers
Delivery	A message published to the channel topic is read independently by each member's client, each tracking its own offset so it can resume after being offline

Two details worth volunteering: ordering must be strict within a channel (a reply must not precede the message it answers) but is meaningless across channels — which is exactly the per-partition ordering Lesson 7 provides. And offsets are what make "unread messages" work: a client that was offline resumes from its last offset rather than losing history.

Functional requirements

Requirement	Detail
Topic creation	Producers can create a topic to group related messages
Write messages	Producers can publish messages to a topic
Subscription	Consumers can subscribe to a topic to receive updates
Read messages	Consumers can read messages from subscribed topics
Retention policy	Users can specify a retention period for messages
Message deletion	The system automatically deletes messages after the retention period expires

The last two requirements are one idea, and it reshapes the storage design

Retention and automatic deletion are the pair that make this not a queue.

In a queue, a consumer's delete call removes the message — deletion is consumption-driven. Here, deletion is time-driven: messages expire on a schedule regardless of whether anyone read them, and no consumer's action removes anything.

Two consequences follow immediately:

Storage is sized by retention, not by backlog. You hold every message for the full period even if all subscribers consumed it in the first second. Retention becomes a capacity decision.
Readers cannot destroy data, so many readers can share one copy safely. That is what makes Lesson 6's single-copy-plus-offsets design possible — and it is precisely the property Lesson 4's design throws away.

Non-functional requirements

Requirement	Detail
Scalability	Handle increasing numbers of topics, producers, and consumers
Availability	Producers and consumers must be able to access the system at all times
Durability	Accepted messages must persist until delivered to all intended subscribers
Fault tolerance	Continue operating despite component failures
Concurrency	Safely handle simultaneous read and write operations

Note the three axes of scalability — they stress different components

"Topics, producers, and consumers" are three independent growth dimensions, and an interviewer may push on which breaks first:

More producers → write throughput. Handled by partitioning (Lesson 6), since partitions absorb writes in parallel.
More consumers → read throughput and metadata. Each consumer needs its own tracked offset, so a million consumers means a million offsets — which is why Lesson 10 puts them in a key-value store rather than a relational database.
More topics → cluster coordination. The cluster manager tracks which broker holds which topic, so topic count grows its state.

Saying "it scales horizontally" addresses none of these specifically. Naming which tier each axis stresses is the senior answer, and it mirrors the two-axis discussion in the messaging queue chapter.

Building blocks we will use

Building block	Used for
Database	Stores metadata, such as subscription details
Distributed messaging queue	Buffers messages sent by producers
Key-value store	Stores transient consumer state

Three chapters you have already built, composed rather than reinvented.

Key takeaway

Four use cases, all sharing one shape: several independent readers over the same stream. Six functional requirements, of which retention plus automatic deletion is the pair that separates this from a queue. And scalability runs on three axes — topics, producers, consumers — each stressing a different tier.

Interview signal by level

Level	What a strong answer sounds like
L4	"Publish, subscribe, read, and it should scale and be available."
L5	Flags retention: "messages expire on a retention policy rather than being deleted when consumed — that's what lets several subscribers read the same copy."
Staff+	Separates the scaling axes and names the operational trap: "topics, producers, and consumers grow independently — producers stress write throughput and want partitions, consumers stress offset metadata and want a key-value store, topics stress the cluster manager. And retention being time-driven means a subscriber offline longer than the window comes back to a silent gap, so I'd alert on consumer lag against retention."

Next: the six calls.