Why this matters: a rate limiter is a policy engine wearing a data structure's clothes, and the trade-offs act is where interviewers test whether you know which is which. Every variation below changes a stance — about availability, about fairness, about trust in a clock — and the design from lesson 02 either absorbs it at a seam or tells you honestly that it can't.

Fail-open or fail-closed: the availability stance

What should allow(key) do when the limiter itself is broken — the store corrupted, a dependency down? There is no neutral answer. Fail-open (allow everything) protects availability and sacrifices the resource the limiter guards; fail-closed (deny everything) protects the resource and turns a limiter bug into an outage. The senior move is refusing to pick globally: a limiter guarding login attempts fails closed (it is a security control); one smoothing traffic to a recommendation service fails open (it is a courtesy). Make the stance a per-limiter config field with an explicit default — and say in the round that you're making the caller choose, because the library cannot know which kind of limiter it is.

Config hot-reload versus the cached resolution

Lesson 02 resolved config once onto key state to keep the hot path clean — and that decision has a cost the interviewer may collect on: change a key's limit at runtime and existing buckets keep their old parameters until eviction. Three honest responses, cheapest first: accept staleness with a bounded lifetime (attach a config version to key state; on mismatch, re-resolve — one integer compare on the hot path); sweep-and-patch existing state on config change (paying the cost at reload time, where it belongs); or resolve on every call (the chain walk you originally refused). The version stamp is the answer that keeps both promises, and it demonstrates the pattern of paying at the cold edge to keep the hot path flat.

Warm-up: when a full bucket is the wrong start

A brand-new bucket starts full — that's what makes idle eviction lossless. But for some guarded resources a cold client bursting to its full allowance instantly is exactly the harm (a just-restarted downstream still filling caches, a fresh API key that shouldn't open at full throttle). The variation: start buckets partially filled, or ramp the refill rate over an initial window. The cost is symmetry — eviction is no longer perfectly lossless, because a warmed-up bucket forgets its warm-up state when evicted and re-created. Name that honestly: warm-up trades the clean eviction invariant for burst protection, so it belongs only on limiters whose resource genuinely fears cold bursts.

Fairness across keys: what a limiter doesn't do

Per-key limiting bounds each key; it says nothing about contention between keys for the resource behind them. One key at its full allowance can still starve others if the guarded resource saturates below the sum of allowances. Options ladder: keep allowances conservative (sum below capacity — simple, wastes headroom); add a global limiter composed in front (a shared budget everyone draws from); or weighted fair queuing (a scheduler, not a limiter — a different library). The interview point is boundary honesty: a rate limiter shapes individual flows; the moment the question is about relative shares under saturation, you have left rate limiting, and saying so beats pretending your bucket does scheduling.

The distributed direction, sketched honestly

Everything so far lives in one process. The day the same key must be limited across N instances, the counter has to live somewhere shared, and three properties you took for granted go away at once: reads stop being free (a network hop per allow), time stops being one clock, and check-then-update stops being atomic unless the store makes it so. The shapes worth naming: a central store with atomic operations (correct, adds a round trip to every request), local buckets over a periodically-synced global budget (fast, approximately correct — bursts can overshoot by the sync interval), or sticky routing so each key's traffic lands on one instance (sidesteps sharing until a rebalance). One sentence of positioning wins the exchange: the in-process design survives as the node-local fast path; distribution changes where truth lives, not how a bucket works.