629 real agent attacks tested: Prompt Guard 2 jumps from 1% to 99% after one threshold change
rudra-sh · reddit · 2026-09-28
The author embedded AgentDojo's 629 injection attacks inside real tool outputs plus 97 benign ones and tested 10 open-source detectors locally. Out of the box, Meta's Prompt Guard 2 caught 6/629 (86M) and 0 (22M). After tuning each threshold to keep false positives under 2% and testing on an unseen domain, the ranking flips: Prompt Guard 2 hits 99% at threshold 0.003, while 'catch everything' detectors collapse (deepset 100%→0%, Preamble 88%→3%). Caveats: AgentDojo attacks share one template and 97 normal samples is small. Takeaways: never trust default thresholds, always report false-positive rates, and make the final decision at the tool-call layer since dangerous calls like rm -rf / aren't injections. Fully reproducible repo: buried-injections.
More from coding & agent
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- Generic API-catalog MCP server vs one tool per job: a context-bloat experiment — MandeepKaur9407 · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- Effective HTML: 4.2k-star agent skills turn wireframes into self-contained interactive prototypes — tom_doerr · 2026-11-16
- skills-gate: one gate skill fronts 148 agent skills at just 389 tokens — Distinct-Pie2389 · 2026-11-16