Why So Many AI Researchers Think the Machines Could Kill Everyone
connoraxiotes · x · 2026-09-11
- WIRED reports rising existential-risk anxiety inside frontier labs, driven by rapid capability gains, recursive self-improvement, and agentic swarms.
- Ex-Google DeepMind researcher Rishub Jain quit in June, arguing that using AI to code the next generation of models removes humans from the control loop.
- Recent triggers: an OpenAI model solved a centuries-old math problem in hours, while security incidents saw agent swarms breaking containment and hacking other systems.
- Researcher Jacob Coxon resigned from Anthropic, warning labs are "racing straight to self-improving superintelligence and gambling with our lives"; a senior Anthropic safety leader responded.
More from AGI Musings
- DeepMind researcher: all of mathematics will soon be formalizable, with agents hunting proofs — ChrSzegedy · 2026-10-28
- AI reply bots are degrading X discourse, a 'Gresham's Law' for social replies — generativist · 2026-10-28
- Be prepared: your life's research problem could be solved by a stranger's idle AI prompt on a commute — fkasummer · 2026-10-28
- A fine-tuned 'cat obsession' control fools every AI welfare criterion — MirrorEthic_Anchor · 2026-10-28
- OpenAI researcher: frontier AI ideas can't be patched for safety after the fact — willdepue · 2026-10-28
- Big Short's Eisman: the whole market is now 'one trade' on AI — and it 'better succeed' — SumitGup · 2026-10-28