Online Learning and Freshness

In one line: the ad inventory changes faster than a nightly retrain, so the model is updated continuously — which is powerful and removes the safety net that batch retraining provides.

Why batch is not enough

Three properties of ads that most machine learning systems do not share.

New entities appear constantly. Campaigns launch, creatives rotate, advertisers arrive. A model trained last night has never seen an ad created this morning and has no opinion about it — while the auction is being asked to price it right now.

Behaviour shifts fast. A news event, a holiday, a competitor's promotion. The relationship between features and click probability moves within hours, and calibration moves faster than ordering does.

The volume is enormous. Billions of impressions a day means a great deal of signal arrives between nightly retrains, and discarding it is expensive in a system where a fraction of a percent of accuracy is real money.

FTRL, and why it fits

The classic online learner for this setting is Follow-The-Regularised-Leader, and the reason it dominated is specific rather than fashionable.

An online learner processes one example at a time and updates immediately, which suits a system where labels stream in continuously. What FTRL adds is sparsity: it produces models where most coefficients are exactly zero.

That matters here more than anywhere else. With billions of possible feature values, a model that keeps a nonzero weight for every value it has ever seen grows without bound, and its memory footprint is the serving cost. FTRL's regularisation drives the useless ones to exactly zero, so they can be dropped from the served model entirely.

It also handles per-feature learning rates naturally — a feature seen millions of times should move less per observation than one seen twice, and FTRL's accumulation gives that for free.

For the deep models, the same principle applies with different machinery: embedding tables are updated incrementally, row by row, as training proceeds. You cannot redeploy a hundred-billion-parameter table hourly, so continuous in-place update is not a preference but the only viable mechanism.

What continuous training costs you

The part candidates skip, and it is where the interview goes.

There is no version to roll back to. A batch model has artifacts. A continuously updated model is a single evolving state, so "revert to yesterday's model" requires having deliberately snapshotted it. Snapshot on a schedule, or a bad hour of data is permanent.

Bad data is absorbed immediately. A logging bug, a bot attack or a broken upstream feature is learned within minutes rather than caught in the next training run. There is no batch validation step standing between the data and production.

Nothing is reproducible. The model that served an impression at 14:32 no longer exists. Debugging "why did this ad win" requires the model state at that moment, which means logging predictions and features rather than hoping to recompute them.

Drift is silent. No retraining event means no natural checkpoint at which someone looks at the metrics.

The safety rails

Four, and proposing them unprompted is what separates a design from a description.

Snapshot and pin. Persist the model state at intervals, and keep enough history to roll back past an incident that took hours to notice.

Validate before ingesting. Check the incoming stream's statistics — click rate, feature distributions, null rates — against recent history, and pause learning rather than absorbing an anomaly. Pausing is nearly always the right default: a slightly stale model is much cheaper than a poisoned one.

Bound the update. Cap how far any parameter can move in a window. A legitimate shift is gradual; a large jump is usually a bug.

Shadow and compare. Run the updating model alongside a pinned one and compare their predictions continuously. Divergence is the earliest available signal that something has gone wrong, and it needs no labels.

Freshness versus sophistication

The trade worth stating explicitly, because it is a genuinely useful and slightly counterintuitive result.

In most machine learning settings a better model architecture beats more frequent retraining. In ads it is often the reverse: a simpler model trained on the last hour frequently outperforms a more sophisticated one trained on last week, because so much of the value is knowing about entities that did not exist a week ago.

This is a scoping question worth asking in an interview — how quickly does the inventory turn over? On a system where campaigns run for months, batch retraining is fine. On one where creatives rotate hourly, freshness dominates and the architecture barely matters.

The hybrid that most systems land on

Not fully online, not fully batch.

The base model is retrained in batch periodically — daily or weekly — on a large window, with full validation, proper evaluation and a deployable artifact. This is where architecture changes and new features ship.

The online layer updates continuously on top of it: recent-behaviour features, per-entity bias terms for new ads, and the calibration mapping. Small, fast, and cheap to reset.

That split gets the freshness where it is needed while keeping the safety of batch training for the part that carries most of the model's capability. And it makes rollback tractable: resetting the online layer to the base model is a bounded, well-defined operation, which is exactly what you want at three in the morning.

Next: spending the advertiser's budget smoothly.