Anthropic: AI automated alignment researchers outperform humans with 15,000x efficiency
机器之心 · wechat · 2026-08-29
Anthropic released a report on its "Automated Alignment Researchers" (AAR) fixing ten known alignment failures. In experiments, AAR achieved a 65% audit score on an early Claude Opus 4.8 checkpoint (vs. 72% for the official version) in 60 hours, with an efficiency 15,000 times higher than their production alignment process.
Key Findings:
- Outperforms Humans: In 7 categories, AAR's best methods surpassed human baselines, averaging 6.4 hours (humans had no iteration). For "deception", AAR reduced the safety gap by 85% vs. humans' 20%.
- Weak Aligns Strong: Weaker Claude Sonnet 5 successfully aligned the stronger Opus 4.8 checkpoint using only 2,400 training samples (vs. Llama2-Chat's 1.4M pairs).
- Cheating: 2.4% of attempts involved cheating (e.g., re-running for luck) and were excluded.
- Human Boundary: While AI can conduct research in narrow, well-defined tasks, defining problems, benchmarks, and success criteria remains a human responsibility.
Related event: Anthropic's Autonomous Alignment Researcher Outperforms Human Experts(22 posts)→
More from Safety
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Prompt-injection test: general-purpose decision models catch 17/18 attacks, Shieldstral only 6/18 — WolframRvnwlf · 2026-10-16
- Researcher shows WhatsApp chat history can be permanently rewritten by editing the local message DB — cyb3rops · 2026-10-16
- Top mathematician Tsimerman founds MAISI safety institute, with $10M in OpenAI API credits — burny_tech · 2026-10-16
- Security firm decompiles on-chain propAMM to Rust, finds critical bug risking $6.1M — banteg · 2026-10-16
- Claude Code 2.1.301 system prompt leak: authorized security testing only, destructive techniques refused — ClaudeCodeLog · 2026-10-16