Operating the Guard Layer
In one line: a guard layer is a policy system that ships continuously, and treating it as code you deploy is the mistake that makes every policy change a week long.
Policy is data, not code
Policies change constantly — a new regulation, a new abuse pattern, a market that requires a different line. If each change is a code change, every one costs a review, a deploy and a rollback plan, and the team stops making them.
So the policy lives in a store, versioned, with the same discipline as a feature definition: an identifier, an owner, a version, examples on both sides of the line, and thresholds per surface.
The logged version is what makes the system explainable. Without it, "why was this blocked?" is unanswerable a week later, when the policy has moved twice.
Every policy needs positive and negative examples attached, because those examples are simultaneously the specification, the eval set and the classifier's training data. A policy without examples is an opinion, and it will be interpreted differently by the classifier, the human reviewer and the appeal handler.
Red teaming is a process, not a launch checklist
A one-off pre-launch exercise tells you about the model you had, against the attacks known then. Both change.
The version that works is continuous and layered:
Automated, every build. A regression suite of known attacks — the jailbreaks, injections and encodings you have already seen — run on every guard or model change. Cheap, and it catches the reintroduction of fixed problems, which is the most common regression.
Automated, exploratory. Attack generation, where a model produces variations on known attacks. Broad coverage, unreliable per-example judgement, useful for finding directions rather than verdicts.
Human, periodic. People who are good at this, given time and the actual product. Consistently finds the things automation does not, and is the only source of genuinely novel attack classes.
External, ongoing. A vulnerability disclosure path with a clear scope and a response commitment. People will find things regardless of whether you offer them a way to tell you.
The human review queue
Automation cannot be the last word, for two reasons: the uncertain band has to go somewhere, and appeals need a decision from a person.
Three inputs feed it — escalations from the guard cascade, user appeals against refusals, and a random sample of both allows and blocks for calibration. The random sample matters most and is dropped first when the queue is busy; without it, you only ever review cases the system already found interesting, and you never discover what it is missing.
Priority is by expected harm times volume, not by score. A moderate-confidence flag on a high-traffic surface deserves attention before a high-confidence flag on a page nobody visits.
Two properties keep the queue honest:
Reviewers see the policy and the examples, not just the content. Consistency between reviewers is a measurable thing, and it collapses when the policy is folklore.
Decisions feed back automatically. A reviewer overturning a block should update the eval set, not just the individual case. Otherwise the same false positive recurs indefinitely and reviewers learn that their work has no effect.
Appeals
If you block people, some blocks are wrong, and the ones that are wrong are exactly the ones you cannot see. An appeal path is both the fair thing and your highest-quality source of false-positive labels.
Three requirements: it must be visible at the point of refusal, it must have a bounded turnaround, and the outcome must reach the person. An appeal path that exists in the settings menu and answers in three weeks generates no signal because nobody uses it.
Monitoring
The dashboard that only counts blocks is the one most teams have, and it cannot show the failure that hurts.
| Signal | What it tells you |
|---|---|
| Block rate per category | Sudden movement means a policy change, an attack, or a broken guard |
| Refusal-then-abandonment | The closest live proxy for over-refusal |
| Appeal rate and overturn rate | A high overturn rate means the threshold is wrong |
| Guard latency, p50 and p99 | The cascade drifting toward the expensive tier |
| Escalation-band volume | Widening uncertainty — often a distribution shift |
| Attack-signature hits by source | Which documents and domains are being used against you |
A block-rate drop is as alarming as a spike. It usually means a guard is failing open, which is invisible in every other metric on the page.
Incident response, when the bug is in a model
The shape differs from an ordinary outage in one important way: you usually cannot fix the model quickly.
So the response has a different ordering. Contain first, at the layer you control — tighten a threshold, disable a tool, restrict a surface, add a deny-rule. Those are configuration changes and they land in minutes.
Then scope it. The prompt-and-completion log is what tells you how many users saw this and which sources triggered it. Without that log there is no scope, only guessing.
Then the durable fix, which is almost never "retrain the model". It is a new guard rule, a permission narrowed, a tool removed, a boundary moved. And finally the regression test, so the next release cannot reintroduce it.
Key takeaway
Keep policy in a versioned store with examples attached, and log the policy version with every decision — that is what makes a block explainable after the policy has moved. Red team continuously and turn every real attack into a regression test plus a note on which layer should have caught it. And build the appeal path, because overturned blocks are the only high-quality false-positive labels you will ever get.
Next: the whole thing, as an interview.