Why AI researchers who believe in extinction risk keep building: inside the exodus
adariostrange · x · 2026-09-18
MARS Magazine examines the wave of AI safety researchers quitting. Jacob Coxon recently resigned from Anthropic, saying firms are 'gambling with our lives', while current Anthropic researcher Evan Hubinger says 'we really do earnestly believe AI could kill all humans.'
- Precedents: Jan Leike and Daniel Kokotajlo left OpenAI in 2024; Anthropic safety lead Mrinank Sharma departed warning 'the world is in peril'; Alex Turner left Google DeepMind after its Pentagon 'killer drones' deal.
- Context: doom narratives trace to Bostrom's 2014 Superintelligence and LessWrong.
- The paradox: in 2023 the CEOs of OpenAI, Anthropic and Google DeepMind agreed extinction risk should rank with pandemics and nuclear war; Dario Amodei puts 'really, really badly' odds at 10–25%.
The piece asks why those who believe in the risk keep building, and offers three main explanations.
More from AGI Musings
- "Causal patterns are all you need": MLStreet Talk debates whether the hard problem exists — burny_tech · 2026-11-02
- Dev says blog traffic fell from 700k to 7k reads as AI-generated job apps flood market — ericelliott_ · 2026-11-02
- Master foundational math to stay irreplaceable, argues mathematician — TivadarDanka · 2026-11-02
- Narayanan: labs racing toward Recursive Self-Improvement may hit Recursive Self-Extinction of their business model — random_walker · 2026-11-02
- Editor fully resigns from JMLR, arguing AI reviews beat rushed humans and journals need a rethink — roydanroy · 2026-11-02
- Alpha School investigation: 17 campuses on probation, Bill Ackman as shadow lead, Holmes involved — benjaminjriley · 2026-11-02