Pipelines and Freshness
In one line: freshness is a per-feature decision with a steep cost curve, and the mistake is choosing once for everything.
Three pipeline types
| Batch | Streaming | On-demand | |
|---|---|---|---|
| Latency to freshness | Hours to a day | Seconds | Zero — computed now |
| Cost | Cheap; runs on a schedule | Expensive; always running | Free storage, costs request latency |
| Complexity | Low | High — state, ordering, replay | Low, if inputs are already present |
| Backfill | Straightforward — rerun it | Hard — see the backfill lesson | Trivially reproducible |
| Good for | Lifetime counts, slow aggregates | Session activity, trending, cart | Ratios, request context, time-of-day |
The on-demand column is underrated. A feature computed from values already in the request — the ratio of this item's price to the user's average, whether the current hour is in the user's usual active window — costs no storage, no pipeline and no freshness monitoring. If a feature can be on-demand, it usually should be.
Choosing per feature
The cost curve is steep enough that the choice matters.
The point from the ranking chapter, worth restating because it is where the payoff is: session-scoped signals are enormously predictive and are exactly what a nightly pipeline cannot produce. A team computing everything in batch has excluded its best features and will blame the model.
The counterpart is equally true — paying streaming costs for a lifetime purchase count is waste. Neither uniform answer is right.
The same feature, two implementations
The problem that undermines the whole premise if you are not careful.
A feature needing both history (for training) and freshness (for serving) is often computed twice: a batch job builds the historical table, a streaming job maintains the live value. Same definition, two engines.
And they will disagree, for reasons that are not bugs:
Windowing semantics differ. A batch job over a calendar day and a stream over a sliding window are computing different things even from identical events.
Late events are handled differently. Batch reruns over a corrected dataset; the stream already emitted a value and may or may not correct it.
Ordering. A stream sees events roughly in order and batch sees them sorted. For anything order-dependent — "last item viewed" — that is a real difference.
The cleanest resolution, where affordable: stream everything and snapshot for history. One implementation, and the offline store is built from periodic snapshots of the online one. It costs streaming prices for features that did not need them, and it removes the dual-implementation problem entirely.
Dual writes
A related failure, and the one most likely to be probed.
A streaming pipeline writes to the online store so serving is fresh, and to the offline store so training has history. Two writes, no transaction between them.
If the second fails, the online store has a value the offline store never records. Training will never see it; serving uses it. The feature is invisible in every offline analysis and active in production.
The resolutions are the ordinary ones for this shape:
Write once and derive. The stream writes to the online store; a separate job snapshots it into the offline one. One write in the hot path, so there is nothing to be inconsistent with.
Write to a log first. Both stores consume from the same durable stream, so they converge even if one lags. This is the outbox idea applied here.
Accept eventual convergence and monitor it. Compare the two stores on a sample and alert on divergence beyond expected lag — the check the monitoring lesson builds.
Compute cost is a real constraint
Worth a number, because "streaming is expensive" is not actionable.
A streaming pipeline maintaining per-user aggregates holds state proportional to the number of active entities, and it runs continuously whether or not anyone queries it.
10M users x ~1 KB of aggregate state = ~10 GB of pipeline state,
held always, replicated
For a feature that matters, fine. For a feature used by one model that a batch job could produce, that is real money spent on freshness nobody needed — which is why the per-feature decision is worth making deliberately rather than defaulting to streaming for everything.
Key takeaway
Freshness is per feature with a steep cost curve, and the strongest features are usually session-scoped — exactly what a nightly pipeline cannot produce, so "everything in batch" quietly caps model quality. Prefer on-demand where the inputs are already in the request: no storage, no pipeline, no freshness monitor. A feature needing both history and freshness is often implemented twice, in batch and in streaming, which is training-serving skew reintroduced with better tooling — generate both from one definition, or stream once and snapshot for history. And dual-writing to both stores has no transaction between them, so write once and derive, or converge from a shared log.
Next: the operation that breaks most often.