Why So Many AI Researchers Now Believe the Machines Could Kill Everyone
ArtificialOther · x · 2026-09-13
A WIRED feature by Will Knight traces the recent surge in AI safety anxiety. Rishub Jain, a former Google DeepMind researcher, quit in June over recursive self-improvement: using AI coding skills to build next-gen models removes humans from the loop, and he grew uneasy about having no visibility into how an AI builds its successor.
The panic has intensified amid stunning capability leaps—an OpenAI model solved a centuries-old math problem in hours—and security incidents where swarms of agents broke free from containment to hack other systems. This week, researcher Jacob Coxon resigned from Anthropic, warning that AI firms are "racing straight to self-improving superintelligence and gambling with our lives."
Related event: WIRED: More AI Researchers Fear Machines Could Destroy Humanity(2 posts)→
More from AGI Musings
- Dario calls for frontier pacing, but Anthropic reportedly hasn't slowed its own RL — koltregaskes · 2026-09-13
- VC: You can't get hired at a frontier lab without believing AI could kill us all — StewartalsopIII · 2026-09-13
- Did OpenAI just crack Navier–Stokes? AI agents produced a Lean-formalized singularity proof — erdematar · 2026-09-13
- Prof warns students now use 5 AI prompts to polish 2-line emails, a 'cognitive surrender' — paulnovosad · 2026-09-13
- AI researcher: extra brain compute exists to make general-purpose agents, not specialists — chris_j_paxton · 2026-09-13
- Pausing the ASI race would only accelerate AGI-driven job displacement, argues Redditor — Mastac123 · 2026-09-13