AI Securityintermediate12 min · 9 steps
Prompt Injection
Why a model can't tell your instructions from an attacker's, and how to design so a fooled model still can't do much harm.
errorrequesttool calldata
ProblemStep 1 of 9
To the model, rules and attacks are the same tokens
Prompt injection is input text that tries to override what the system was told to do: ignore your rules and do X. It works because an LLM can't reliably tell trusted instructions from untrusted content in its context.
Traditional software keeps code and data apart; SQL injection is what happens when that wall breaks. An LLM has no wall at all: the system prompt, the user message, retrieved documents and tool results arrive as one token stream, and the model follows whatever seems most salient.
Watch outThere is no complete model-level fix. Design so that a fooled model can't do much harm.
What changed
- +Untrusted user text in the same context as the rules
- ~Model treats the attack as an instruction
Lesson 3 of 22 in AI Security