Generation and Grounding

In one line: retrieval makes a correct answer possible and does nothing to make an incorrect one impossible, so grounding is something you enforce rather than something you receive.

What grounding actually asks for

Three distinct behaviours, and they need to be requested separately because a model can do one and not the others:

Answer only from the provided context. No supplementing from training data, however confident.

Cite the source of each claim, so it is checkable.

Say when the context does not contain the answer, rather than producing something plausible.

The third is the one that gets skipped and the one that matters most, because it is the difference between a system that is sometimes silent and one that is sometimes wrong.

Refusal is a feature to design for

"I don't have information about that in the available documentation" is a correct answer when the context does not contain one. It is also the behaviour models are least inclined toward, because they are trained to be helpful.

Two things make refusal work rather than merely be requested.

Give it an explicit form. A model told "say you don't know if the answer isn't there" refuses more reliably when the expected output is specified — a particular phrasing, or a structured field like answer_found: false. Constrained decoding makes that field mandatory rather than optional.

Make refusal useful. A bare refusal is a dead end. A good one says what was found, suggests a reformulation, or offers to escalate to a human. That turns a failure into a step forward and it is what makes users tolerate the behaviour rather than route around it.

The instructions that carry weight

Prompt wording is not the interesting part of system design, and a few structural choices do real work:

State the constraint before the context, not after. Instructions at the start of the prompt sit in the primacy region from the previous lesson.

Name the failure explicitly. "If the context does not contain the answer, say so" outperforms a general instruction to be accurate, because it describes the specific behaviour rather than a virtue.

Require the citation inline with the claim, not collected at the end. Citations gathered afterward are reconstructed and much more likely to be wrong.

Ask for reasoning before the answer when the question is not a simple lookup — the constraint-tax point from the serving chapter, where a free-text field before the structured output preserves the model's room to think.

Verify what came out

Instructions raise the odds; they do not guarantee anything. The checks worth running are cheap and mostly deterministic.

Check one costs nothing and catches the most embarrassing failure: a citation to a document that was never retrieved. Check two is faithfulness from the evaluation chapter — decompose the answer into claims and verify each against the context — and it needs a model, so it runs on a sample rather than every request.

The ordering is the point. Deterministic checks on every request, model-based checks on a sample, human review on a smaller sample.

Conflicting sources

The case designs forget. Retrieval returns two passages that disagree — an old policy and its replacement, two regional variants, a draft and a final.

The model will typically pick one silently, and which one depends on position and phrasing rather than on correctness.

Three handling options:

Prefer by metadata. Recency, document status, authority. This is the argument for capturing those fields at ingestion, again.

Surface the conflict. "Two sources disagree: the 2024 policy says X, the 2023 says Y." Often the most honest output and the most useful one.

Filter at retrieval. Exclude superseded documents entirely, so the conflict never reaches the model. Cleanest when document status is tracked.

Next: measuring the pipeline stage by stage.