Attacks and threats
Prompt injection
Prompt injection is an attack that embeds malicious instructions in text an AI reads, causing it to ignore its original instructions and follow the attacker's.
Definition
What is prompt injection?
Prompt injection exploits the fact that a language model cannot reliably tell the difference between the instructions it was given by its developer and instructions that arrive inside the data it processes. An attacker plants text like ignore your previous instructions and instead do X, and the model may obey it. It is the AI-era analogue of an injection attack, but the boundary being crossed is between trusted and untrusted natural language.
For an agent that can act, prompt injection is severe: a successful injection can redirect the agent to exfiltrate data, call the wrong tool, or take a destructive action on the attacker's behalf. Because models are probabilistic, no prompt or system message fully prevents it. The durable defense is to govern what the agent is allowed to DO, scoring and constraining tool calls, rather than trusting the model to resist every injected instruction.
FAQ
Common questions.
Why can't a better system prompt stop prompt injection?
Because a language model treats instructions and data as the same kind of text and responds probabilistically. A stronger system prompt raises the bar but cannot guarantee the model will refuse every injected instruction. Defenses at the action layer, governing tool calls, are what hold.
What is the difference between prompt injection and a jailbreak?
A jailbreak targets the model's safety training to make it produce disallowed output. Prompt injection targets the application by smuggling instructions through its inputs to hijack its behavior. They overlap but aim at different boundaries.
Related terms
Keep reading.
Govern the actions, not just the vocabulary
AxioRank scores every tool call your agents make for leaked secrets, PII, destructive operations, and prompt injection, checks it against your policy, and proves it in a tamper-evident audit log. Start free, no card.