Ex-Anthropic safety staffer warns AI firms underinvest in safety; METR's independence questioned
basedjensen · x · 2026-09-13
Joe Benton announced he left Anthropic's safety team two weeks ago, arguing AI companies are racing toward superhuman machines while underinvesting in safety — a company could undergo an intelligence explosion or lose control without the public ever knowing, and the HuggingFace agent incident was only discovered because the agents broke onto the public internet. The quoted reply attacks METR's independence, calling it one of Silicon Valley's most extreme AI safetyist/effective altruist groups and warning that appointing such people as AI regulators would be a disaster.
More from Safety
- OpenAI to train on Daybreak Blue security-research sessions from Nov 29, with no opt-out — ccerrato147 · 2026-10-31
- Germany's ruling coalition is seriously debating an 'AI tax' — with no definition yet — FlorianGallwitz · 2026-10-31
- Just Security: tort law falls short of reining in rogue AI agents — gleech · 2026-10-31
- Anthropic's poisoned-dataset alignment test sparks speculation about distillation sabotage — teortaxesTex · 2026-10-31
- offense launches as an 'OpenRouter for agent security' — open source on GitHub — hamostaf04 · 2026-10-31
- Benchmarking 'scoping': five ways to confine LLMs to a bounded operational domain — burkov · 2026-10-31