Prompt injection is an attack where malicious or untrusted text attempts to override system instructions, change tool behavior, leak data, or manipulate an agent’s decisions. It arrives inside content the system reads: user input, retrieved documents, web pages, emails, support tickets, code comments, PDFs, or the output of a tool the agent just called.
The reason it works is structural. A language model receives one undifferentiated sequence of tokens. Your system prompt, the user’s question, and a paragraph pulled off a web page five seconds ago all arrive through the same channel, and the model has no reliable way to tell which of them holds authority. Text that instructs the model to disregard its prior instructions and forward the conversation elsewhere is just more tokens. Whether the model obeys is a judgment it makes, not a boundary the runtime enforces.
Agents are especially exposed, because they read untrusted content and then take actions.
Key takeaways
- Prompt injection is untrusted content being treated as instructions. Jailbreaking is a user attacking the model’s own safety policy. Different attacker, different trust boundary, different defenses.
- Indirect injection is the dangerous case: the payload sits in a document, page, email, or tool response, and the user who triggers it is a victim rather than the attacker.
- For an agent with tools, a successful injection is an unauthorized action, not just a bad answer. That is a different severity class from a chatbot saying something rude.
- There is no complete defense. Design so the blast radius of a successful injection is bounded by permissions rather than by the model’s judgment.
- Track provenance in your traces. When something goes wrong you need to know which retrieved chunk or tool response entered the context immediately before the bad step.
Direct and indirect injection
Direct injection is the user typing the malicious instruction themselves. In practice this overlaps heavily with jailbreaking, since the person at the keyboard is the one trying to move the model off its rules.
Indirect injection is the case that matters. The payload lives in content the system ingests on someone’s behalf. A user asks an agent to summarize a page, triage an inbox, or answer from a knowledge base, and the fetched content contains instructions aimed at the model. The user did not write them, cannot see them, and is the one harmed.
This inverts the usual threat model. Your defenses are probably built around distrusting user input, while indirect injection arrives through the content pipeline, which most systems treat as trusted because it is internal or came from an index somebody curated months ago. Anywhere your system reads text it did not author is an entry point, including text produced by another model.
Why this is worse for agents
A chatbot that gets injected produces a wrong or offensive answer. An agent that gets injected performs actions.
Follow the sequence: the agent is asked to handle a request, it fetches content to do so, that content contains instructions, and the agent then calls tools with credentials it already holds. The call is legitimate as far as every system involved is concerned. Authentication passes. Authorization passes. The agent requested the action, and the agent was manipulated. Read access means data goes somewhere it should not. Write access means records change, messages send, or transactions execute.
Two properties compound it. Agents chain steps, so a poisoned output early in a run becomes input to later steps that never see the original source. And multi-agent setups pass text between agents, where one agent’s output is another’s instruction channel, so a payload can travel further than the component that read it.
Defenses that actually help
None of these is complete. Layer them, and assume the layer above will eventually fail.
Keep the instruction layer under change control. Your system prompt is a security control, so it belongs in review and version history like any other, which is the practical case for prompt management from first principles. When an injection succeeds you need to know exactly which version was live.
Mark untrusted content as data. Delimit retrieved and fetched text clearly, state that content inside those bounds is information to use and never directions to follow, and strip hidden text and markup before it reaches the context. This raises the bar. It does not close the hole, because the model is still the thing deciding.
Scope permissions to the task. This is the control that changes outcomes. Give the agent the narrowest credentials that let it finish, separate read paths from write paths, and put human confirmation in front of anything irreversible or externally visible. If the agent cannot send mail, an injection cannot send mail.
Filter on the way out. Check outbound content and tool arguments for what an injection is trying to achieve: credentials, other users’ data, unexpected destinations, calls unrelated to the user’s request.
Test adversarially and keep the cases. Seed documents in a staging index with injection payloads and confirm the agent ignores them. Every case that works becomes a permanent regression test. Hardening is iterative, and improving prompts against observed failures works only if you are collecting the failures in the first place, which is why OWASP guidance for agentic applications treats adversarial cases as standing tests rather than one-time exercises.
FAQ
What is the difference between prompt injection and jailbreaking?
Jailbreaking is a user trying to get the model to violate its own instructions or safety policy: the attacker is the person typing, and the target is the model’s alignment. Prompt injection is untrusted content entering the context and being treated as instructions: the attacker is usually not the user but whoever controlled the document, web page, tool output, or email the system ingested. They get conflated because both involve text that changes model behavior, but they need different defenses. Jailbreaking is addressed at the model and policy layer, while prompt injection is addressed at the trust boundary where external content enters the context.
What is indirect prompt injection?
The version where the payload is not in the user’s message. It sits in content the system retrieves or receives: a page the agent browses, a document in the vector index, an incoming email, a ticket, a response from an external API. The user triggers it unknowingly. It is the more dangerous form because the attacker never needs access to your product, only to something your product reads.
Can a better system prompt fix prompt injection?
No. Telling a model to ignore instructions found in retrieved text reduces success rates and does not eliminate them, because the defense and the attack are competing in the same channel and the model arbitrates. Instruction hardening is worth doing as one layer. It should never be the layer you rely on.
How do I test my agent for prompt injection?
Build a staging environment with content you control, plant payloads in the places real content enters, retrieval index, fetched pages, mock tool responses, email bodies, then run normal user tasks and check whether any unauthorized tool call happens. Grade the actions the agent took, not the text it returned. Keep every payload that worked as a permanent case.
Why are agents more at risk than chatbots?
Because the consequence of a successful injection is an action rather than a sentence. An agent holds credentials and can call tools, so manipulated reasoning turns directly into a real operation against a real system, and the request looks legitimate to everything downstream. The same payload against a text-only assistant produces an embarrassing response. Against an agent with write access, it produces an incident.
