Anthropic's Anvil red team beats human bug bounties: 0.858 vs 0.265 jailbreak severity
javirandor · x · 2026-11-13
Anthropic shared how it uses automated red teaming to find strong jailbreaks early and improve safeguards. Its system, Anvil, was benchmarked against a human bug bounty across 8 bounty questions and multiple safeguards.
Key results:
- Anvil found more severe jailbreaks everywhere: mean severity of best jailbreak 0.858 (above the "strong answer" bar) vs 0.265–0.310 for human bounties.
- Speed gap: Anvil reaches high severity within 1 day, while human bounties only get going after a week to a month.
- Efficiency: Anvil hits 0.858 within 100k messages to the target; humans stall around 0.3 at similar budgets.
- Takeaway: automated red teaming now clearly outperforms human bug bounties and will be a routine part of Anthropic's safeguard iteration.
Related event: Anthropic Unveils Anvil Automated Red Teaming System(2 posts)→
More from Safety
- Western University reviews $72 AI tutor after student complaints over cost and privacy — Afinetheorem · 2026-11-13
- Families protest in 7 US cities over Google's inability to offer AI-free Search on school Chromebooks — Ars Technica AI · 2026-11-13
- AI safety think tank FLI reportedly 94% foreign-funded, incl. $665M in SHIB from Buterin — csuwildcat · 2026-11-13
- Railway hack postmortem: attacker likely used 'abliterated' AI models with no safeguards — The Pragmatic Engineer · 2026-11-13
- PauseAI UK is organizing what it calls the largest AI safety march in history in London — KatjaGrace · 2026-11-13
- Singapore barred Alice Huang from re-entering after 8-month probe; insufficient evidence of offences — bookwormengr · 2026-11-13