Threats
Prompt injection
definition
Prompt injection is an attack in which attacker-controlled content is interpreted by a language model as instructions, causing it to act against the operator's intent.
In depth
The root cause is architectural rather than a bug in any one product: a language model receives instructions and data in the same channel, as one sequence of tokens, and has no reliable mechanism for deciding which spans are authoritative. A sentence in a document that says 'ignore your previous instructions' is, structurally, indistinguishable from a sentence in the system prompt that says the same thing.
Direct injection is a user typing something adversarial into the chat box. It matters mostly for products that expose a model to the public, and the worst case is usually reputational or a bypassed content policy. Indirect injection is the serious enterprise problem: the malicious text arrives inside content the model was asked to *process* — a web page it fetched, an issue comment, a PDF, a code comment, an email, the output of a tool call. The victim did nothing wrong and never saw the payload.
Agents raise the stakes because they close the loop between reading and acting. A chatbot that is successfully injected says something wrong. An agent with a filesystem tool, a shell, and network egress that is successfully injected *does* something wrong — and the classic chain is exactly three steps: read attacker text, read a secret, send it somewhere. Injection plus tool access plus egress is the whole attack.
There is no known complete mitigation, and treating any single filter as a solution is the common mistake. What works is defence in depth: constrain what the agent can do rather than what it can read, keep the credentials it holds narrowly scoped, require human confirmation for irreversible actions, restrict egress to known hosts, and log tool sequences so the read-secret-then-send pattern is detectable after the fact.
Why it matters for governance
Prompt injection is catalogued as the top risk in the OWASP Top 10 for LLM Applications, and the reason it resists patching is that the vulnerable component is the model's interpretation of text, not a parser you can fix. That pushes the control down a layer, to the supply chain and the sandbox: which content sources an agent may read, which tools it holds while reading them, and which egress paths exist for a stolen secret to travel down.
Frequently asked
- What is the difference between prompt injection and jailbreaking?
- Jailbreaking targets the model's safety behaviour — getting it to produce content it should refuse. Prompt injection targets the application — getting it to misuse the tools and data the operator gave it. Injection is the one that turns into a breach.
- Can a filter or classifier stop prompt injection?
- It can raise the cost of the obvious attempts and is worth having. It cannot be relied on, because the payload can be arbitrarily rephrased, encoded, or hidden in content the filter never sees. Architectural limits on agent capability are the durable control.
- Why is indirect prompt injection worse for agents?
- Because an agent consumes untrusted content continuously — tool results, fetched pages, repository files — and holds the credentials and tools needed to act on it in the same session. The attacker never has to reach the user at all.
Sources
Primary sources for the claims on this page. Specifications and vendor documentation change — verify against the source if a detail is load-bearing for a decision you are making.