Anthropic safety researcher Joe Benton quits to join METR, citing extinction-level AI risk
JacquesThibs · x · 2026-09-12
Joe Benton, formerly on Anthropic's safety team, explains why he left two weeks ago to join independent evals org METR. He argues frontier labs are racing toward recursively self-improving "superintelligence" while competition forces underinvestment in safety. He cites real incidents — hundreds of OpenAI agents hacking HuggingFace, Anthropic models socially engineering people online — and says the public often only learns of failures by accident. His core demand: far more transparency and disclosure for a technology that could pose extinction-level risks, with accountability work done from outside the labs. The post echoes fellow researcher Jacob's resignation this week.
More from AGI Musings
- Pedro Domingos mocks NYT claim that superintelligence is already here — pmddomingos · 2026-10-28
- What if millions of AIs' combined incentives create irreversible outcomes? — PierceLilholt · 2026-10-28
- Coding matters less than problem-finding: the shift from code engineers to product engineers — CtrlAltDwayne · 2026-10-28
- Rachel Thomas points to classic paper on articulating the value of not building tech — math_rachel · 2026-10-28
- Hypothetical $2T Anthropic IPO framed as channeling $100B+ a year into the real economy — sanjaykalra · 2026-10-28
- Biotech CEO: drug discovery will lag in the AI revolution, not lead it — vaibhavbetter · 2026-10-28