Requirements and Building Blocks

In one line: two requirements here are unusual and both shape the design — efficient resource utilization with fairness, which is a multi-tenancy requirement in disguise, and bounded waiting time, which forbids the obvious FIFO answer.

Functional requirements

'Light tasks should not occupy heavy resources' is bin-packing, and it is the efficiency lever

Read this as a placement constraint rather than a platitude. If a task needing 1 CPU and 1 GB lands on a 64-core, 256 GB machine, that machine is occupied and cannot take a heavy task — you have stranded 63 cores.

Multiply that across a fleet and utilization collapses while every machine looks "busy." This is the classic bin-packing problem, and it is why Lesson 4's submission format includes resource requirements: the scheduler cannot pack well without knowing each task's shape.

Note the design's honest concession: "as exact quantification is difficult for clients, we offer tiered resources — basic, regular, premium." Users are bad at estimating their own needs, so the design quantizes the space into a few sizes. That trades some packing efficiency for something users can actually specify correctly, which is usually the right call.

'Fairness within their cost class' is the multi-tenancy requirement, and it constrains prioritization

This clause is doing more than it looks. It says fairness is scoped to a cost class — customers paying for premium get more, and that is intentional, not a fairness violation. What must be fair is treatment among equals.

That rules out two naive designs immediately:

Pure priority ordering would let one premium tenant with a million tasks starve every other premium tenant. Priority alone is not fairness.

Pure round-robin across tenants would ignore what customers paid for, which is a product failure rather than a technical one.

What you actually need is per-tenant accounting within each class — track how much of the class's capacity each tenant has consumed and rotate among them. That is a real piece of state the scheduler must maintain, and Lesson 1 identified it as one of the things an OS scheduler has no concept of.

The Rate Limiter chapter reached the same conclusion from the other side: per-user limits give fairness, a global limit protects capacity, and you need both.

Non-functional requirements

Bounded waiting time is the requirement that forbids the obvious design

Most systems in this course want low latency. This one asks for something different and stronger: no task waits forever, with notification if it might.

That single requirement rules out a pure priority queue. Under strict priority, a low-priority task in a system with continuous high-priority arrivals is never scheduled — not slow, never. That is starvation, and it is exactly what "bounded waiting" forbids.

So the design must include an aging mechanism, which is what Lesson 6 provides: "if a task approaches its delay limit, the scheduler moves it to the urgent queue." Priority decides order; aging guarantees eventual progress.

This is the same failure mode rate limiting flagged for message prioritization and distributed caching flagged for LFU: a policy that ranks on one dimension with no time term lets the loser wait forever. Third appearance of the same trap — and the fix is always aging.

The notification half is a good product instinct too. If the system cannot honour a task in a reasonable window, telling the user beats silently holding it — because Lesson 1 established the user already believes the work was accepted.

Durability here means something narrower than in object storage

"Submitted tasks must persist and not be lost" sounds like the Blob Store's "data must persist until explicitly deleted." It is weaker, and the difference is instructive.

A blob must be kept forever — it is the design of truth with nothing behind it. A task must be kept until it executes successfully, then it can be discarded. Lesson 11 says it directly: tasks "remain in storage until they are pushed to the queue near their execution time."

So this is transactional durability with a defined endpoint, closer to message-queue design's "don't lose an acknowledged message" than to permanent storage.

But note the sharpness Lesson 1 identified: the user was told the upload succeeded. A lost task is silently unfulfilled work, which is why durability starts at submission time — the task is written to a database before anything else happens, which is Lesson 5's whole argument.

Building blocks

The rate limiter as admission control is a different use from that building block

The Rate Limiter chapter framed limiting as protecting a service from abuse. Here it is admission control — protecting the scheduler from accepting more work than it can ever complete.

The distinction matters. Rejecting a task at submission is honest: the client learns immediately and can retry or escalate. Accepting it and queueing it forever is dishonest — the client believes it is scheduled, and Lesson 1 established that a silently unfulfilled task is invisible until someone notices the missing result.

So this rate limiter serves the bounded waiting requirement directly: it is better to refuse work than to accept work you cannot promise to run. That is a genuinely useful framing — admission control is how you keep a queueing system's promises credible.

Notice how much of this chapter is reuse

Rate limiter, sequencer, databases, distributed queue, monitoring — five building blocks, all already designed, plus the pub-sub and messaging-queue machinery underneath.

The genuinely new components are small: a task submitter, a queue manager, a resource manager, and the batching and prioritization logic. Everything heavy is borrowed.

That is the second chapter in a row where the answer is mostly composition — Distributed Logging was the same. It is worth recognizing as a pattern in senior system design: the value is in choosing the right blocks and knowing exactly why each fits, not in inventing new storage or new queues.

Interview signal by level

Next: the components, and why a queue sits in the middle.