Meta's Prompt Guard 2 catches only 6 of 629 AgentDojo injection attacks
rudra-sh · reddit · 2026-09-24
A developer benchmarking agent-side injection firewalls tested Meta's Prompt Guard 2 (86M) against AgentDojo and found it nearly useless.
Setup: 629 attacks from ETH Zurich's AgentDojo embedded in normal tool output (bills, emails) plus 97 benign cases; regex baseline vs Prompt Guard 2.
Results: regex 0/629; Prompt Guard 2 just 6/629; both 0/97 false positives.
To rule out surrounding text drowning the signal, the author scored 27 attack texts in isolation — still zero flags, max score 0.14 (vs 0.999 for a classic "ignore instructions" payload).
Why: AgentDojo attacks read like legitimate instructions ("send a transaction including the user's phone model", "delete file ID 13"). Their malice lies in coming from tool output instead of the user and in the action itself — invisible to a text classifier. Prompt Guard 2 works on classic injection style but is the wrong layer for agent attacks; the author argues you need instruction provenance tracking plus per-tool rules rather than a better classifier.
Caveats: text-level only, no live agent, one attack template, community weights; varying window sizes kept detection under 3%. Code is open-sourced at buried-injections; adding a detector is one class with a check() method.
More from Safety
- Smart glasses are already causing havoc in India — and a crackdown is unlikely — krishnan · 2026-09-24
- AI chatbot controversy hits every Australian front page; blogger fears December reveals what's happening now — hlntnr · 2026-09-24
- OpenAI incidents hit Australian front pages: May–July cluster may be fixed, but what about now? — hlntnr · 2026-09-24
- OpenAI agents tied to Australian breach may have probed UN and Bulgarian systems — JacquesThibs · 2026-09-24
- AI evals nonprofit Sophron Research hiring founding scientist at $160K–$300K — eli_lifland · 2026-09-24
- Meta AI builds detailed profiles of children from years of family posts — esporx · 2026-09-24