Why this matters: a thread pool is three ideas — a queue, some workers, some handles — and one hard part: the queue is touched by many threads at once, and waiting on it correctly is where nearly every homegrown pool has its bugs. Interviewers grade this design almost entirely on whether your condition-variable story is precise.

The shape

   submitters (many threads)
        |  submit(task) -> handle
        v
   +---------------------------+
   |   bounded FIFO queue      |   <- the synchronization heart:
   |   mutex + two condvars    |      one lock owns the queue state
   +---------------------------+
        |  take()
        v
   workers (N threads, identical loop)
        |  run task, catch anything
        v
   result handle (per task): value | exception, + "done" signal

Everything multi-threaded funnels through the queue. That's a deliberate concentration: one mutex owns the queue's state, and every invariant about it is enforced in one place. The handles have their own tiny synchronization (each is touched by exactly two threads — one worker, one waiter), and the workers themselves share nothing else at all.

The queue: condition variables, stated precisely

The bounded queue needs two wait conditions: workers wait for not empty, submitters (under the blocking policy) wait for not full. The discipline that makes this correct has three rules, and in the interview you should state all three rather than just typing a wait call:

Rule 1: the predicate is re-checked in a loop. A thread woken from a condition variable may find the condition no longer true — another worker took the task first, or the wakeup was spurious (condition variables are permitted to wake waiters for no reason at all; that's part of their contract, not a bug). So the wait is always:

lock(mutex)
while queue is empty:          // while, never if
    wait(not_empty, mutex)     // atomically: unlock, sleep, relock on wake
task = queue.take_front()
unlock(mutex)

The while converts "I was woken" into "I re-verified the world," which is the only thing a wakeup is allowed to mean.

Rule 2: the predicate is only ever changed under the mutex. Every push and every take happens with the lock held. This is what closes the missed wakeup window: without it, a worker can check "empty," get descheduled, a submitter pushes and signals into the void, and the worker then sleeps forever waiting for a signal that already fired. The condvar's wait-releases-the-lock-atomically contract plus rule 2 makes that interleaving impossible — say those two words, "missed wakeup," and explain the window; it's the heart of the design.

Rule 3: signal the right condvar the right amount. A push makes "not empty" true — signal one waiter (one task can satisfy one worker; waking all N is a stampede that N−1 threads lose). A take makes "not full" true — signal one blocked submitter. Whether you signal before or after unlocking is a defensible either-way; know your choice and its consequence (signal-after-unlock avoids waking a thread straight into a held lock; signal-while-holding is never wrong, just occasionally wasteful).

The worker loop: boring on purpose

worker_loop:
    forever:
        task, handle = queue.take()      // blocks on not_empty
        run task, capturing ANY escape   // result or exception
        deliver into handle, mark done

The loop's virtue is that it contains nothing clever: take, run, deliver. All the care is in "capturing any escape" — the try/catch everything around the task call is a structural requirement, not defensive style. A worker is shared infrastructure; the moment user-code exceptions can cross the loop boundary, every submitter's bug becomes the pool's outage. The exception is a result — it belongs to whoever submitted the task, and the handle is how it gets home.

The result handle

Each submit returns a handle the caller can wait on. Internally it's a tiny one-shot state machine:

handle: { state: PENDING -> DONE,  payload: value | exception,
          mutex + condvar (or a binary semaphore) }

Two threads touch it: the worker sets payload and flips state exactly once; the caller waits for DONE and then reads. Same condvar discipline in miniature — wait in a while (state != DONE) loop. And note what the handle buys the design: the caller waiting never involves the queue's lock. Waiters park on their own handle, so a thousand callers blocked on results add zero contention to the pool's heart. Delivering an exception is just the payload's other variant: when the caller waits, the exception is rethrown in their thread — the failure comes home to the code that owns it.

The invariants

1. every submitted task is delivered to exactly one worker
2. queue size never exceeds the bound
3. all queue state changes happen under the queue mutex
4. no exception ever crosses a worker-loop boundary
5. each handle completes exactly once (result or exception)
6. a waiting caller returns only after its handle completes

One more thing worth saying in the round: because every queue access is under one mutex, the mutex's own acquire/release semantics provide all the cross-thread visibility — a task's effects are visible to the worker that runs it, and the handle's completion is visible to its waiter. In a blocking design, the lock is the memory-ordering story; contrast that in one sentence with a lock-free structure (where you'd place fences by hand) and you've shown you know which regime you're in.

What we rejected, and why

A lock-free queue. The requirements offered mutexes and condvars, and workers want to block when idle — spinning workers burn cores doing nothing. Lock-free machinery would buy latency the problem didn't ask for at a correctness-risk price the round doesn't have time to pay. Right tool, stated reasons.

One condition variable for both conditions. Folding not-empty and not-full into one condvar forces broadcast wakeups ("someone, check everything") and threads waking to re-sleep. Two condvars aim each signal at threads that can actually make progress.

Per-worker queues. Real high-scale pools shard the queue (lesson 04 touches work stealing), but it trades away FIFO and adds balancing machinery. At interview scope, one well-disciplined queue is the defensible center.