The Feedback Loop

In one line: the system chooses what gets shown, what gets shown determines what gets logged, and what gets logged trains the system — so its mistakes are the only thing it cannot learn about.

The property that makes recommenders different

A fraud model sees transactions it did not choose. A translation model sees text it did not write. A recommender sees only interactions with items it decided to show.

That closes a loop, and closed loops have dynamics.

The upper loop is the popularity spiral. The lower one is worse and less discussed: a wrong negative belief is permanently self-preserving, because the only evidence that would overturn it requires showing the item, which the belief prevents.

An item the model wrongly scores low is not eventually corrected. It is buried, and it stays buried.

Three ways it manifests

Popularity concentration. Exposure concentrates on a shrinking set. Each impression was individually the best available prediction; the aggregate is a catalogue where most items are effectively invisible. This is a supply problem before it is a user problem — creators who receive no distribution stop supplying.

Narrowing. A user shows mild interest in a topic, receives more of it, engages more, receives still more. Nobody decided to narrow their feed; the loop did it. Whether this is a problem depends on the product, and it is always worth naming: a music recommender narrowing to a favourite genre may be exactly right, while a news feed doing the same is a different matter.

Metric inflation. Retrain, and offline metrics improve — because the new model agrees with the old one about what to show, and the evaluation data was generated by the old one. You are measuring agreement, not quality, and the number goes up regardless.

That third one is the trap in an interview. "Our offline metrics improved every retrain" is not evidence of improvement in a closed-loop system.

What actually breaks the loop

Three mechanisms, and only three.

Exploration

Deliberately show items the ranking would not have chosen, and log what happens. This is the only source of evidence about the model's negative beliefs, which is why it is the only genuine fix — everything else redistributes information you already have.

It costs immediate engagement, which makes it a budget decision rather than a technique. Its own lesson follows.

Randomised holdouts

Keep a small slice of traffic on a non-personalised or randomly-ordered experience. It gives an unbiased reference: how does the population behave when the system is not steering them?

Two things it buys. An unbiased estimate of item quality for evaluation. And a check on whether personalisation is helping at all — occasionally, on some surfaces, the answer is that it is not, and only a holdout can tell you.

Signals from outside the loop

Any evidence not produced by your own ranking. Surveys, because the question is asked rather than inferred from what you showed. Search queries, because the user chose the item. External popularity. Explicit follows and subscriptions.

These are sparse and expensive, and their value is exactly that they are outside the loop.

What does not break it

Worth being explicit, because these get proposed and they do not work.

A diversity term in the loss. It changes what is shown, so it does help a little. It does not generate evidence about items outside the current candidate set, because the loss only operates over candidates that were retrieved.

Re-weighting popular items down. Corrects the estimate for items you have data on. Says nothing about items you have none on.

Retraining more often. Tightens the loop. If anything, it accelerates the dynamics.

A bigger model. Fits the biased data better.

The common thread: the problem is missing data, and no modelling change creates data.

Detecting it

Since the loop hides itself, monitor for it directly.

Watch the trend, not the value. There is no correct Gini coefficient; a Gini that rises every month is a system closing in on itself.

Saying this in an interview

This is one of the highest-value things you can raise unprompted, because most designs describe a training pipeline as if the data were given. Something like:

This system trains on data it generated, so its own mistakes are the one thing it can't learn about. An item it wrongly scores low is never shown, so no evidence ever contradicts that. I'd budget explicit exploration for that reason, keep a randomised holdout as an unbiased reference, and monitor catalogue coverage — because offline metrics will improve either way.

Next: cold start, which is the feedback loop at its worst.