Output Guards and the Streaming Problem

In one line: the output guard is the only one that sees what was actually produced, and streaming means it has to decide before the thing it is judging is finished.

Why the output side is where the real checks live

The input guard judges intent from a request. The output guard judges a result. Several failures are only visible there.

A hallucinated fact looks like a normal question going in. A leaked record looks like a normal lookup. A response that is technically compliant but off-brand is invisible until it exists. And an injected instruction that succeeded produces its damage in the output, not the input.

Four checks are worth naming, and they are different mechanisms:

Groundedness is a retrieval check, not a truth check

Worth being precise, because candidates overclaim here. You cannot verify that a statement is true. You can verify that it is supported by the context you supplied, which is a different and tractable question: split the response into claims, and for each, ask whether a retrieved passage entails it.

That catches the failure that actually matters in a retrieval system — the model asserting something the sources do not say. It does not catch a wrong source, and it says nothing about a response with no sources at all.

Cheaper proxy, and often enough: require citations, then verify mechanically that each cited passage exists, that it was actually retrieved for this request, and that it is textually similar to the sentence citing it. That is a string-and-lookup check, not a model call, and it catches fabricated citations — which are common.

The streaming problem

Here is the part most designs skip.

Streaming exists because time-to-first-token dominates perceived latency. But an output guard needs a complete response to judge it. Those two facts are in direct conflict, and there is no configuration that satisfies both.

Buffer the whole response and you have thrown away streaming — the user waits for the full generation plus the guard. Stream freely and the guard is judging text the user has already read. Once a chunk crosses the client boundary, a refusal cannot recall it.

The practical answer is a bounded exposure window

Release at a granularity that is large enough to judge and small enough not to feel buffered. A sentence is the natural unit: it is semantically complete, so a classifier has something to work with, and one sentence of lag is barely perceptible.

The cost is stated honestly as a window, not as a solution. With sentence-level release you are exposed to at most one sentence of unsafe content, and only if the violation is confined to a sentence the guard scored as safe. Larger chunks give better context — a hallucination is easier to spot across a paragraph — at proportionally more lag.

Two failure modes survive, and naming them is the depth signal:

Cross-chunk violations. Content that is harmful only in combination passes every chunk-level check. This is the multi-turn problem again, one level down.

The retraction problem. If sentence four fails, sentences one to three are already on screen. You need a defined behaviour — replace the message, or append a correction — and either way the user saw something. Replacement is usually right, and it needs to be built into the client protocol from the start rather than added later.

Never execute model output

A rule, not a preference, and OWASP keeps it in the top ten because it keeps happening.

Model output is untrusted input to whatever consumes it. Rendering it as HTML is a cross-site scripting vector. Interpolating it into a query is SQL injection. Passing it to a shell is command injection. The model may have been persuaded to produce exactly that payload by content it read three steps earlier.

The controls are the ones the industry already knows: encode at the sink, parameterise queries, validate against a schema before use, and never eval. A generated tool call is validated as data — the tool name is in the allowed set, arguments match the schema, values are in range — before anything runs.

What to do when the guard fires

The decision is not binary, and treating it as such wastes information.

Block and explain. The user gets a refusal that says what happened. Better than a generic error, which reads as a bug and generates a support ticket.

Regenerate once. For borderline scores, a single retry with a stronger instruction often produces an acceptable response. Bound it at one attempt — a retry loop against a persistent violation is a cost attack on yourself.

Redact and release. Where the violation is localised, such as a leaked identifier, remove it and serve the rest.

Serve a safe completion. For high-harm categories, a fixed response is safer than anything generated, because it cannot itself be manipulated.

Whichever fires, log the input, the generation, the guard scores and the action. That log is your only evidence at incident time, and it is the training data for the next version of the guard.

Next: PII, and why detecting it is the easy half.