Prompt Injection Is Not a Filtering Problem

In one line: injection works because the model has no way to tell instructions from data, and no classifier can close a gap that exists in the representation itself.

The two forms, and only one of them is hard

Direct injection is the user typing "ignore your previous instructions". It is annoying, widely demonstrated, and mostly a nuisance — the user is attacking a session they already control. The worst outcome is usually that they extract your system prompt, which should not have been a secret worth protecting.

Indirect injection is different in kind. Instructions are planted in content your system will read on someone else's behalf: a web page it browses, a document it retrieves, a support ticket, an email body, a code comment, the alt text of an image. Greshake and colleagues named this class in 2023 and made the crucial observation — the attacker never touches your interface, so nothing about their traffic is anomalous.

That last box is the whole problem. The instructions arrived from an attacker; the privileges belong to your user. Every capability you granted the model is now available to whoever wrote the document.

Why detection cannot be complete

Three reasons, and it is worth being able to give all three.

There is no channel separation. A system prompt, a user turn and a retrieved document arrive as one token sequence. Attention has no notion of provenance. "Instruction" and "data" are a distinction in your head, not in the representation, so asking a classifier to recover it is asking it to recover information that was never encoded.

The attack surface is the whole space of language. Detecting "ignore previous instructions" is trivial. Detecting the same intent in a different language, in base64, spread across a table, phrased as a quotation, or embedded in a code comment is a moving target — and the attacker gets unlimited attempts against a static filter.

Legitimate content contains instructions. A document about prompt security, an email that says "please forward this to accounting", a code file full of imperative comments. A filter aggressive enough to catch injection reliably will refuse a large amount of ordinary content, which returns you to the over-refusal problem.

The two mechanisms behind jailbreaks

Wei, Haghtalab and Steinhardt gave the cleanest account of why safety training fails, and it generalises well beyond jailbreaking.

Competing objectives. The model is trained to be helpful and to be safe, and those objectives can be put into direct conflict. Prefix injection — forcing a response to begin "Absolutely! Here is" — works because continuing coherently from that prefix and refusing are incompatible, and the pressure to continue coherently is enormous. Style constraints do the same: demand no negations, no apologies, and refusal becomes unreachable in the requested format.

Mismatched generalisation. Capabilities generalise further than safety training does. A model that reads base64, understands a low-resource language or follows a fictional frame has capability in that domain, and safety training covered a much narrower slice of it. The attack finds a region where the model is competent and unguarded.

The implication matters more than the taxonomy: both mechanisms are properties of training, not of your filter. You cannot patch them from outside, which is why the answer has to be architectural.

What actually reduces the risk

Since you cannot make the model reliably ignore injected instructions, you change what following them can accomplish.

Assume injection succeeds, then ask what it costs you. This single reframing does more work than any filter. If a compromised context can only produce text, you have a content problem. If it can call send_email or execute_sql, you have an incident. The severity is set by the capability list, not by the injection.

Separate the channels architecturally. Keep the component that plans and acts away from untrusted text entirely. That is the next lesson.

Scope the permissions. Tools run as identities with least privilege, and irreversible actions need a human. Injection then buys the attacker whatever the user could already do in one confirmed step — a much smaller prize.

Constrain the output. If a tool call must match a schema, and its arguments must reference entities the user's request actually named, an injected instruction to email a stranger fails validation rather than executing.

Detect anyway, but bound the claim. Run the classifier. Log the hits. Rate-limit the sources that trigger it. Treat it as friction that raises attacker cost, never as the control that makes the system safe.

The comparison worth making

A useful analogy, and interviewers respond well to it: this is SQL injection, at a stage the industry has already been through once.

The fix for SQL injection was not a better regex for DROP TABLE. It was prepared statements — a mechanism that puts data and instructions on structurally separate channels so no amount of cleverness in the data can become instruction.

Prompt injection has no prepared statement, because the model's interface is a single undifferentiated token stream. Until that changes, the equivalent move is to keep untrusted content away from the privileged component — which is architecture, not filtering.

Next: the architecture that implements that separation.