AI Securityintermediate13 min · 10 steps
Red Teaming
Attack your own AI system on purpose — people plus automated tools, against the full pipeline — and turn “we think it's secure” into an attack success rate tracked every release.
critical ASR?
requesttool callerrordatasignal
metrics are illustrativeProblemStep 1 of 10
A release adds a powerful tool. Is it safe?
The next release of the IT assistant lets the action agent request group memberships — the kind of change that grants access to systems.
The team has guardrails, scoped tools and approvals, and believes it's secure. But a belief is not evidence: nobody has tried to break this tool the way an attacker would.
Watch outEvery capability change re-opens the question. The tool you add today is the attack surface tomorrow.
What changed
- +New request_group tool
- ~? Attack success rate unknown
Lesson 22 of 22 in AI Security