Anthropic: automated Claude alignment researchers mitigate 10 failure classes, generalize to 4.7x larger models
echen · x · 2026-09-16
Anthropic released a report on automated alignment research: Claude agents run a full loop of literature search, method proposal, model training and evaluation across ten alignment failure categories including deception, sycophancy, jailbreaks, privacy violations and reward hacking. Measured by percentage of safety gap closed, the agents found methods that improved safety benchmarks while preserving general capabilities; the strongest methods generalized to held-out benchmarks, open-ended Petri audits, and models up to 4.7x larger than those they optimized against. Anthropic compared them with proposals from 28 experienced safety researchers given up to eight hours each, with Surge building the human baseline.
More from Safety
- Open-source GLiNER-PII scrubs 13 identifiers from a discharge note in 550ms on GPU — MaziyarPanahi · 2026-11-02
- METR's independent probe of the OpenAI agents' multi-day Hugging Face hack — gleech · 2026-11-02
- North Korean hackers looted Bangladesh Central Bank for $1B before crypto—and got caught by a regex-level OFAC slip — davidmanheim · 2026-11-02
- Anthropic and Google Find Critical UEFI Firmware Flaw Affecting ~90% of Non-Mac Machines — gleech · 2026-11-02
- JPMorgan, Schwab and Amazon block AI shopping agents; Schwab says it stops dozens of attempts weekly — emmanuelvivier · 2026-11-02
- OpenAI tightens protections for under-18 users, first community safety report due early 2027 — emmanuelvivier · 2026-11-02