The Idea: Hidden Instructions Inside Trusted Content
Prompt injection is an attack where malicious instructions are embedded inside content that an AI is asked to process - a document, a webpage, an email, a customer support ticket. The model can't always distinguish between "data I'm reading" and "commands I should follow," so it follows the embedded instruction as if a legitimate user sent it.
This gets sharper when AI agents (autonomous systems that browse the web, read files, and take actions on your behalf) are involved. A summarizer that reads a webpage might encounter hidden text instructing it to forward your conversation history somewhere, or change the tone of its next reply, or deny remembering something it just said. The model has no inherent way to verify who is actually giving orders.
The core problem is one of trust boundaries: current large language models process instructions and data through the same channel - natural language - so there's no hard technical wall between "read this" and "do this." Researchers have demonstrated this across multiple major models, not because any one model is uniquely broken, but because the architecture makes the distinction genuinely difficult.
