Evaluation Is the Loop's Ground Truth

In one line: the loop cannot see the task, only the score, so the evaluation is not a way of checking the system — it is the system's definition of what good means.

Evaluation appears twice in an AI system and the two uses are not the same object. In a report, a metric summarises. In a loop, a metric is a target that something is actively searching against. The second use has requirements the first one never needed.

Three tiers, cheapest first

Push as much as possible down to the first tier. Not because programmatic checks are more insightful — they are less — but because they are the only tier the optimiser cannot talk its way past. A schema either validates or it does not.

The mistake is treating the tiers as a fallback chain, where the judge handles whatever the checks could not. They are better used as a gate sequence: a candidate that fails a programmatic check never reaches the judge, which saves budget and, more importantly, stops the judge from being asked to weigh a hard violation against a soft improvement.

What changes when a judge sits inside a loop

A judge used for reporting has known biases — position, verbosity, self-preference — and those are handled with randomised ordering, rubrics and calibration against human labels.

A judge used as an optimisation target has all of that plus one new property: something is now searching for its defects on purpose. Not adversarially in intent, but the effect is identical. Every systematic quirk becomes a direction the loop can move in.

The mitigations that follow from that:

• Score against a rubric with explicit disqualifiers rather than a single quality number, because a scalar invites trading a real defect against surface polish.
• Hold the judge's own version fixed for the duration of an optimisation run, and version it. A judge that changed mid-run makes every comparison in that run incoherent.
• Periodically re-validate the judge against human labels, and treat agreement decay as a stopping condition for the loop rather than a note in a dashboard.
• Never let the loop optimise the rubric.

The metric has to include what you do not want lost

A single quality score gives the loop one direction and no constraints, and it will trade anything not measured for a little more of the thing measured. So the score the loop optimises should be a primary metric plus a set of guardrails that are checked rather than traded: cost per request, p95 latency, refusal rate, safety violations, output length, coverage across input segments.

Segment coverage deserves its own mention. An aggregate score can improve while a minority segment gets worse, and aggregates are exactly what an optimiser will exploit — the cheapest way to raise a mean is often to sacrifice a small slice. Evaluate per segment and require no segment to regress beyond a threshold.

Preconditions for switching it on

The loop should not run until the evaluation can pass a few plain tests. It must separate artifacts you already know differ. Its measurement noise must be smaller than the effect size you care about — worth measuring directly by scoring the same artifact twice and looking at the spread. It must cover the segments that matter. And it must have at least one tier the optimiser cannot influence.

A loop switched on before those hold does not fail loudly. It runs, promotes, and reports improvement, which is the worst available outcome.

Next: what happens as the optimisation pressure goes up.