Free preview

Versioning and Schema Evolution

In one line: a model is a function of a specific feature definition, so changing that definition changes the model — silently, and without redeploying it.

The failure

A feature's definition is improved. "Preferred category" moves from a 90-day window to 30 days, because 30 tracks recent interest better.

The pipeline is updated. The online store starts serving the new value. And every model trained on the old definition is now receiving a different quantity under the same name.

Quality degrades, no deployment happened, and the change that caused it was in a different repository owned by a different team. This is the hardest kind of regression to attribute, and it is entirely preventable.

Version the definition, not just the values

The rule: a change in what a feature means is a new version, not an edit.

ChangeBreaking?Handling
Fix a bug producing wrong valuesNew version; retrain and compare before switching
Change the window or aggregationNew version — the meaning changed
Change the source table, same semanticsNo, if values matchValidate on overlap, then swap in place
Add a new featureAdditive; existing models unaffected
Widen a typeUsually notCheck downstream tolerance
Change null handlingYes, and it looks harmlessNew version — the distribution moved

The last row deserves attention. Switching from dropping nulls to imputing zero looks like an implementation detail and changes the feature's distribution, which changes what the model learned. It reads as non-breaking and is not.

The general test: would a model trained on the old values behave differently on the new ones? If yes, it is breaking, regardless of how small the code change was.

Models declare what they need

The mechanism that makes versioning useful rather than merely recorded.

A deployed model carries a manifest of the features and versions it was trained on, and the serving path requests those specific versions. Then:

A new version does not affect existing models. They keep requesting what they trained on.

Migration is explicit. Retrain on the new version, compare, deploy the new model. Two deliberate steps.

Rollback works. Reverting a model reverts the features it uses, because the manifest is part of the model rather than of the environment.

Deprecation

Versioning creates a cost nobody plans for: old versions accumulate, and every one is a pipeline that runs and storage that fills.

The lifecycle worth stating:

Mark deprecated in the registry, with a reason and a replacement, so anyone discovering it sees the status.

Find the consumers — this is where usage counts earn their place. Deprecating a feature you cannot enumerate the users of is a guess.

Give a migration window, long enough for teams to retrain, which is weeks rather than days.

Stop computing, keep the data, then delete later. Halting the pipeline reclaims most of the cost immediately, and keeping the historical values allows a rollback.

A registry without deprecation becomes an accumulating list of features nobody dares delete, which is a mess that gets worse and never better.

Schema evolution in the sources

The version above the feature: your sources change too, and the store sits downstream of decisions it does not control.

A column is renamed. The pipeline breaks loudly. This is the good case.

A column's semantics change while the name stays. The pipeline continues, computing a different thing under the same name. Silent, and only distribution monitoring catches it.

A column becomes nullable. Values that were always present start being absent. If the transformation drops nulls, the population being aggregated silently shrinks.

The structural mitigation is a contract between the source and the store — an agreed schema with an owner, so a change is a negotiation rather than a surprise. That is organisational, and it is the only thing that reliably works. Failing that, distribution monitoring on the feature is the detector of last resort, which the next lesson builds.

Key takeaway

A model is a function of a specific feature definition, so changing that definition changes the model without anyone deploying it — the hardest regression to attribute, in a repository owned by someone else. Version definitions rather than editing them, and use the test "would a model trained on the old values behave differently on the new ones?", which catches null-handling changes that look harmless. Have models declare the feature versions they were trained on, so behaviour is closed over the model rather than over the environment and rollback works. Plan deprecation, since old versions accumulate as running pipelines. And agree schema contracts with sources, because a renamed column breaks loudly while changed semantics do not break at all.

Next: the serving path, and what to do when a feature is missing.

Enjoying the preview?

Create a free account to unlock the rest of this course, the in-browser judge, and live AI mock interviews.

Sign up free to continue