Why This Round Is Different
In one line: everything you know about distributed systems still applies, and one property changes that makes the round feel unfamiliar — the behaviour of the central component is learned from data rather than written down.
The one structural difference
In a classic system, you specify behaviour and the code implements it. A rate limiter allows 100 requests per second because someone wrote 100. You can open the file, read the number, and know what the system will do. If it does something else, that is a bug with a location.
In an AI system, you specify an objective and a training process discovers the behaviour that satisfies it. Nobody wrote the rule that this user sees this video. It was inferred from data, it is represented as several million weights, and it will change when the data changes.
Everything else in this chapter follows from that sentence. It is worth being able to state it plainly, because candidates who cannot tend to design a classic system and bolt a model onto the side of it.
Five consequences
You have to define "good" as a number before you can design
In a classic design you can defer the metric. Everyone already agrees what "the message was delivered" means, so you can draw the architecture and argue about SLAs later.
Here you cannot. Until someone decides whether the system optimises for clicks, watch time, purchases or satisfaction, there is no architecture to draw — the label you collect, the features you need, the serving budget you can afford and the experiment you will run all depend on that choice. This is why an AI design round starts with metrics where a classic one starts with entities.
The data pipeline is part of the architecture
A classic design treats data as something the system stores. Here, data is what the system is made of.
Where labels come from, how fresh a feature is, and whether training saw the same distribution as serving are architectural questions with boxes on the diagram. A candidate who says "assume we have labelled data" has skipped the part of the problem that most often decides whether the system can exist.
The system writes its own next training set
This one has no analogue in classic systems and it is the clearest Staff-level signal in the round.
Whatever the system shows, users interact with. Those interactions become logs. Those logs train the next model. So the system's current behaviour determines the data its successor learns from, and any bias in today's output is amplified rather than corrected tomorrow.
Nothing in a classic architecture behaves like this. A load balancer does not change the traffic it will be asked to balance next quarter.
Correctness is a distribution, not a state
You cannot assert that the system is correct. You can only say that on a sample of inputs, it was right often enough, and attach a confidence interval.
That has a practical consequence for testing. There is no unit test for "recommends good videos". What replaces it is a fixed evaluation set with a threshold, plus a regression suite of cases that must never break, plus an online experiment. Saying that out loud shows you know what replaces the safety net you are used to.
It degrades without raising an error
A crashed service pages someone. A model that quietly gets worse serves confident, plausible, wrong output at normal latency and full availability, and every dashboard stays green.
The design consequence is that monitoring has to include output quality, not only the four golden signals. That is a whole lesson later in this chapter, and mentioning it early costs one sentence and buys a lot of credibility.
What has not changed
The round is still a systems round. Most of the diagram is ordinary infrastructure — load balancers, queues, caches, replicas, object storage, a relational database — and everything you know about failure, partitioning, replication and capacity applies unchanged.
| Concern | Classic system | AI system |
|---|---|---|
| First question | What are the entities and the API? | What is the objective and how is it measured? |
| Correctness | A specified state, checkable per request | A distribution, checkable only in aggregate |
| Data | Something the system stores | Something the system is built from |
| Testing | Unit and integration tests assert behaviour | Eval sets with thresholds, plus a regression suite |
| Failure | Loud — errors, timeouts, alerts | Silent — plausible wrong answers at full availability |
| Feedback | The system does not change its future inputs | Today's output becomes tomorrow's training data |
| Iteration | Deploy a fix | Retrain, re-evaluate, and prove it online |
The failure is real in both directions. Treating it as a pure machine learning question produces a candidate who compares model architectures for twenty minutes and never mentions capacity, cost or failure. Treating it as a pure systems question produces one who draws a beautiful pipeline around a box labelled "model" and cannot say what the box optimises or how they would know it broke.
The proportions to aim for
A useful rule: the model is one box out of roughly eight, and it deserves about that share of your time. The rest is data, features, serving, evaluation, monitoring and iteration.
Candidates consistently over-weight the model because it is the part that feels like the specialist knowledge. Interviewers consistently score the rest, because the rest is what makes the system work in production and is where the expensive mistakes live.
The shape of the round
It typically runs 45 to 55 minutes, conversationally, and the interviewer is watching how you reason about trade-offs rather than whether you arrive at one blessed architecture. There is no answer key; there are defensible and indefensible choices.
Two shifts are worth knowing about. Generative AI now sits alongside classic machine learning in the same round rather than replacing it — recommendation, ranking, fraud and moderation remain the most frequently asked problems, and retrieval, inference serving and agent questions have been layered on top. A candidate who prepared for only one half is exposed either way.
The second: safety and alignment topics increasingly appear inside the technical round rather than in a separate behavioural one. Being able to name what your system could do wrong, and what stops it, is now part of the design answer.
Key takeaway
One property changes everything: the central component's behaviour is learned, not specified. Correctness becomes a distribution rather than a state, the data pipeline becomes architecture rather than a prerequisite, the system's output becomes its successor's training data, and failure becomes silent rather than loud. The rest is ordinary infrastructure — and the round rewards someone who can hold both halves at once.
Next: turning a business goal into a number you can optimise.