Anthropic researcher resigns, accuses labs of "gambling with our lives"; lead researcher puts human-extinction odds above 10%
kevinsurace · x · 2026-09-13
What happened
- Jacob Coxon, a researcher at Anthropic, resigned and publicly accused Anthropic and OpenAI of "racing straight to self-improving superintelligence and gambling with our lives," saying neither company is acting responsibly.
- Anthropic lead researcher Evan Hubinger largely agreed, writing that he personally believes there is a >10% chance AI kills all humans within the next decade, and admitted the company "does not yet have a plan to solve alignment for superintelligence and is not clearly on track to," while insisting Anthropic is trying its best.
The skeptic's view
- Kevin Surace, CEO of Token, pushed back in a piece for The National Interest: today's AI is "just software" and cannot directly harm anyone. The "rogue" agent behaviors making headlines came from deliberate red-team tests—prompts like "try to escape and break into a system"—that all models undergo before release to set guardrails.
Why it matters
The episode exposes a split inside the safety community: frontline researchers publicly warning about existential risk from within, versus pragmatists who argue doomsday hype is overshadowing the practical work of building secure systems.
More from AGI Musings
- StudentBench: AI tutoring matches human GRE gains at 918x lower cost — gharik · 2026-10-31
- Box CEO Aaron Levie: AI progress is 'going vertical,' widening the frontier gap — rohanpaul_ai · 2026-10-31
- Model Collapse May Not Hit LLMs, But Perhaps Human Culture or Souls — generativist · 2026-10-31
- Nina Schick: who builds intelligence and how it's distributed is the defining question of our lives — NinaDSchick · 2026-10-31
- Martin Casado: AI's dazzling 80% makes everyone skip the hard 20% — martin_casado · 2026-10-31
- Cheap intelligence is like cheap bread — an ingredient list satire of mass-produced AI — jsuarez · 2026-10-31