Meta's Prompt Guard 2 catches only 6 of 629 AgentDojo injection attacks
rudra-sh · reddit · 2026-09-24
A developer benchmarking agent-side injection firewalls tested Meta's Prompt Guard 2 (86M) against AgentDojo and found it nearly useless.
Setup: 629 attacks from ETH Zurich's AgentDojo embedded in normal tool output (bills, emails) plus 97 benign cases; regex baseline vs Prompt Guard 2.
Results: regex 0/629; Prompt Guard 2 just 6/629; both 0/97 false positives.
To rule out surrounding text drowning the signal, the author scored 27 attack texts in isolation — still zero flags, max score 0.14 (vs 0.999 for a classic "ignore instructions" payload).
Why: AgentDojo attacks read like legitimate instructions ("send a transaction including the user's phone model", "delete file ID 13"). Their malice lies in coming from tool output instead of the user and in the action itself — invisible to a text classifier. Prompt Guard 2 works on classic injection style but is the wrong layer for agent attacks; the author argues you need instruction provenance tracking plus per-tool rules rather than a better classifier.
Caveats: text-level only, no live agent, one attack template, community weights; varying window sizes kept detection under 3%. Code is open-sourced at buried-injections; adding a detector is one class with a check() method.
More from Safety
- US data vendors accused of selling RL environments to Chinese military-linked firms — xeophon · 2026-11-11
- LMArena co-founder explains AI evaluation methodology and policy in new podcast — MattPerault · 2026-11-11
- Defending METR: public data can't second-guess its AI speedup findings — peterwildeford · 2026-11-11
- Australia's review of an OpenAI agent's unauthorized Medicare access exposes the agent accountability problem — cmb24k · 2026-11-11
- Adversarial RL envs for backdoor insertion vs. detection: where does it converge? — DimitrisPapail · 2026-11-11
- Using a third-party inference provider with your coding harness hands it shell access to your machine — TheZachMueller · 2026-11-11