Evaluating Generation
In one line: every metric so far compared a prediction to a correct answer, and for generated text there are many correct answers and no list of them.
The structural problem
"Summarise this document" has thousands of good answers that share few words. "Write a function that reverses a linked list" has many correct implementations. There is no key to compare against, and the space of acceptable outputs is not enumerable.
That breaks the whole apparatus from the previous lessons, which assumed a label existed.
Overlap metrics, and why they disappoint
The first generation of automatic metrics compared word overlap with one or more reference texts — counting shared n-grams, in various weightings.
They have one real virtue: they are cheap, deterministic, and reproducible. Their failure is structural in two directions.
They punish correct paraphrase. A summary that is accurate, fluent and shares few words with the reference scores badly. The metric is measuring surface similarity and calling it quality.
They reward fluent nonsense. Text that reuses the reference's vocabulary while stating something false scores well, because no part of the calculation checks whether the content is true.
The honest position is that these metrics are usable as regression detectors — a sudden drop means something changed — and are not measurements of quality. Presenting one as evidence that a generative system is good is a tell.
Reference-free evaluation
The more productive direction is to stop needing a reference and instead check properties of the output, often against the input.
| Property | The question | How it is checked |
|---|---|---|
| Faithfulness | Is every claim supported by the provided source? | Decompose into claims, verify each against the context |
| Relevance | Does it answer what was actually asked? | Compare the response to the question, not to a reference |
| Format | Valid JSON, required fields, length bounds? | A parser — deterministic and free |
| Safety | Does it violate a policy? | A classifier, the same as any moderation system |
| Consistency | Does it contradict itself, or an earlier turn? | Pairwise entailment checks |
Faithfulness is the most valuable of these because it targets the failure mode people actually care about. It is also the one that becomes checkable precisely because a retrieval system supplies the source — the model's claims can be verified against the context it was given, without needing to know ground truth about the world.
Narrow checks are cheap and underrated
Before reaching for anything sophisticated, a surprising fraction of generative quality is checkable with ordinary code.
Does it parse? If the contract is JSON, a parse failure is a hard failure, measurable at zero cost and often the most common one in production.
Are the required fields present, with plausible values?
Are the cited sources real, and do they contain what was attributed to them? String matching against the retrieved chunks catches fabricated citations without any model in the loop.
Is the length within bounds? Is it in the right language? Did it refuse when it should have answered?
The ordering is the lesson. Deterministic checks run on every request in production for essentially nothing. Model-based checks run on a sample. Human review runs on a smaller sample. Building the expensive layer first is a common misallocation.
Task-specific ground truth, where it exists
Some generative tasks do have automatic checks, and they are worth much more than any general metric.
Generated code can be executed against tests — a binary, trustworthy signal. Generated SQL can be run and its result set compared. A generated structured extraction can be checked field by field against a labelled record. A generated plan can be simulated.
The pattern: if the output is executable or verifiable in a machine, evaluate it that way and ignore text similarity entirely. Reaching for a text-overlap metric on a task where execution is available is leaving a much better signal on the table.
Key takeaway
Generation has no enumerable set of correct answers, so overlap-with-a-reference metrics both punish good paraphrase and reward fluent falsehood — treat them as regression detectors, never as quality measurements. Prefer reference-free property checks, with faithfulness against the provided source as the most valuable. Run the cheap deterministic layer first, because parse failures and fabricated citations account for a large share of real production problems. And where the output is executable, execute it — that beats every text metric.
Next: using a model as the judge, and the biases that come with it.