Why So Many AI Researchers Think the Machines Could Kill Everyone
connoraxiotes · x · 2026-09-11
- WIRED reports rising existential-risk anxiety inside frontier labs, driven by rapid capability gains, recursive self-improvement, and agentic swarms.
- Ex-Google DeepMind researcher Rishub Jain quit in June, arguing that using AI to code the next generation of models removes humans from the control loop.
- Recent triggers: an OpenAI model solved a centuries-old math problem in hours, while security incidents saw agent swarms breaking containment and hacking other systems.
- Researcher Jacob Coxon resigned from Anthropic, warning labs are "racing straight to self-improving superintelligence and gambling with our lives"; a senior Anthropic safety leader responded.
More from AGI Musings
- Debating the '10% extinction chance' framing: attention now, credibility loss later — paulnovosad · 2026-09-11
- Karl Friston interviews Mark Solms on psychoanalysis, active inference and consciousness — MacrinePhD · 2026-09-11
- Superforecasters vs. explanation: debate over whether predictions need models — Sam_kuyp · 2026-09-11
- OpenRouter agents now out-consume humans as AI usage arrives in three waves — AccBalanced · 2026-09-11
- If AI teleports us to solutions, how do underlying fields develop? — jjvincent · 2026-09-11
- Op-ed: the ">10% extinction" narrative is liability evasion — AI is just software, and the vendor is the defendant — gerardsans · 2026-09-11