Evaluation

In one line: these tables have three columns instead of two — requirement, component, and how — and that third column is what makes them checkable. Three rows do not survive the check.

Functional requirements

Three columns instead of two, and the third is what makes a compliance table useful

Most tables in this module pair a requirement with a component. These add how the architecture satisfies it — and that column is where a claim becomes falsifiable.

Compare:

TWO COLUMNS:   "Low latency | Regional deployment + cache"
               -> unarguable, and unverifiable

THREE COLUMNS: "...requests are routed to the nearest region, cached completions
                bypass GPU inference, and batching improves GPU throughput"
               -> three specific mechanisms, each checkable

The three warnings above exist precisely because the table is specific enough to check against the rest of the design. A vaguer table would have passed unchallenged, which is the point.

A compliance row that cannot be falsified is documentation, not evaluation.

Non-functional requirements

The latency row credits the cache that barely hits and omits the one that closes the budget

Low latency: "Requests are routed to the nearest region, cached completions bypass GPU inference, and batching improves GPU throughput."

All three are true and the emphasis is inverted.

REGIONAL ROUTING:   essential — cross-region RTT alone can spend the budget
BATCHING:           essential — but continuous, not static, or it adds queueing
RESPONSE CACHE:     ~0% hit rate on inline completion (Lesson 3)
PREFIX KV REUSE:    1,300 ms -> 10 ms.  NOT LISTED.

The row credits a Redis cache whose key contains the file content and cursor position — which change on every keystroke — while omitting the mechanism that turns an impossible prefill into a trivial one.

And the omission has a consequence beyond bookkeeping. Because prefix reuse is not named as a latency mechanism, its routing requirement is never surfaced: the design says load balancers "distribute traffic across GPU clusters", which is exactly what breaks it. A design that does not know which mechanism delivers its SLO cannot protect that mechanism.

Name the mechanism that actually produces the guarantee, or the architecture will be changed in ways that quietly remove it.

GPU auto-scaling is slower than the failure it is meant to absorb

Scalability: "GPU nodes are added or removed dynamically based on traffic patterns across time zones."

True over hours and misleading over minutes, and the mismatch matters here more than in most systems.

Traffic spike (a region's workday begins)   -> minutes to hours   ✅ autoscaling works
Load surge within a region                   -> seconds            🔴 too slow
Node failure                                 -> immediate          🔴 far too slow

Acquiring a GPU node is not acquiring a container: capacity may be constrained, the model must be loaded — 2 TB of weights is not a fast copy — and the KV cache starts empty, so every session routed there pays a cold prefill.

Cold GPU node joins the fleet
  -> load weights          minutes
  -> empty prefix cache    every request is a 1,300 ms cold prefill
  -> it makes latency WORSE before it makes capacity better

That is the property to state: a newly added inference node degrades P99 before it helps.

So the real capacity strategy is the one the chapter lists under cost efficiency — quantization, caching, and the debounce — plus provisioned headroom sized to the predictable time-zone curve. Autoscaling smooths the daily wave; it does not absorb a spike.

When warm-up takes minutes and the budget is milliseconds, autoscaling is a cost optimization, not an availability mechanism.

The fault-tolerance row is the best in the module's AI chapters

Fault tolerance: "If inference fails, the system returns an empty suggestion while maintaining IDE responsiveness."

Short, specific, and correct — and it is the rare compliance row that names a product decision rather than an infrastructure one.

What makes it strong is that it accepts a real loss. The system does not pretend to be available; it says when we cannot answer, we say nothing, and it justifies that against the user experience: an error toast interrupts, and interruption is the thing the product exists to avoid.

Compare with the availability rows elsewhere in this module — "redundancy through duplicated elements", "implementing disaster recovery protocols" — which describe intentions and never say what the user sees when it fails.

A fault-tolerance requirement should specify what the user experiences during failure, not only which components are redundant. This one does.

The refinement from Lesson 5 stands: the policy is right for getCompletion and wrong for explainCode, where the developer clicked a button and silence is a broken feature.

What the evaluation omits

Five things, each covered earlier:

Which cache closes the budget. Prefix KV reuse is the mechanism; the response cache is credited instead (Lesson 3).

Session affinity. Prefix reuse requires it, and the design specifies undirected distribution — the two are incompatible (Lesson 6).

Fill-in-the-middle. A cursor has code after it, and nothing in the design uses the suffix (Lesson 4).

Where the code comes to rest. Telemetry stores the payload and the vector index is a copy of the repository; neither is addressed by TLS or encryption at rest (Lesson 7).

Cost. A stated requirement, never computed — and the 30–40% GPU saving is quoted against a baseline that is 72× too large (Lesson 2).

Next: the interview walkthrough.