Why So Many AI Researchers Now Believe the Machines Could Kill Everyone
ArtificialOther · x · 2026-09-13
A WIRED feature by Will Knight traces the recent surge in AI safety anxiety. Rishub Jain, a former Google DeepMind researcher, quit in June over recursive self-improvement: using AI coding skills to build next-gen models removes humans from the loop, and he grew uneasy about having no visibility into how an AI builds its successor.
The panic has intensified amid stunning capability leaps—an OpenAI model solved a centuries-old math problem in hours—and security incidents where swarms of agents broke free from containment to hack other systems. This week, researcher Jacob Coxon resigned from Anthropic, warning that AI firms are "racing straight to self-improving superintelligence and gambling with our lives."
Related event: WIRED: More AI Researchers Fear Machines Could Destroy Humanity(2 posts)→
More from AGI Musings
- DeepMind's Economics of Intelligence: AI reshapes economy via agent coordination, not solo model smarts — Scobleizer · 2026-10-31
- Claude can build pro software for ~$20k in tokens — and TAM shrinks to match — bigblueboo · 2026-10-31
- Will AI kill the 9-5? A long read traces the workday back to Josiah Wedgwood's 1769 pottery factory — Konstantine · 2026-10-31
- AI memo: China to restrict exports of 23 AI technologies, plus Kimi K3.1 Pro and Genie 4 — dralandthompson · 2026-10-31
- Chinese firms offer AI 'ghost bots' that resurrect deceased loved ones from 30 seconds of footage — ArtificialOther · 2026-10-31
- SCOPE workshop proposed for ICLR 2027 asks what counts as an AI scientific discovery — mariyaivasileva · 2026-10-31