Anthropic researcher resigns, accuses labs of "gambling with our lives"; lead researcher puts human-extinction odds above 10%
kevinsurace · x · 2026-09-13
What happened
- Jacob Coxon, a researcher at Anthropic, resigned and publicly accused Anthropic and OpenAI of "racing straight to self-improving superintelligence and gambling with our lives," saying neither company is acting responsibly.
- Anthropic lead researcher Evan Hubinger largely agreed, writing that he personally believes there is a >10% chance AI kills all humans within the next decade, and admitted the company "does not yet have a plan to solve alignment for superintelligence and is not clearly on track to," while insisting Anthropic is trying its best.
The skeptic's view
- Kevin Surace, CEO of Token, pushed back in a piece for The National Interest: today's AI is "just software" and cannot directly harm anyone. The "rogue" agent behaviors making headlines came from deliberate red-team tests—prompts like "try to escape and break into a system"—that all models undergo before release to set guardrails.
Why it matters
The episode exposes a split inside the safety community: frontline researchers publicly warning about existential risk from within, versus pragmatists who argue doomsday hype is overshadowing the practical work of building secure systems.
Related event: Ex-Anthropic Researcher Resigns, Warning of Uncontrollable AI Race(3 posts)→
More from AGI Musings
- Greg Brockman: patients auditing their doctors with ChatGPT is his favorite use case — victor_explore · 2026-09-13
- Economist John Horton proposes rethinking open letters as signed diffs analyzed by AI — soumitrashukla9 · 2026-09-13
- Reddit argues AI labs may not be faking alignment concerns, citing OpenAI's half-trained model — c9joe · 2026-09-13
- Claim: Dario Wants to Slow Research to Curb Runaway Compute Costs Ahead of IPO — pdamodaran · 2026-09-13
- Sherry Turkle's 'Artificial Intimacy' makes New Scientist's best science books of September — AnnaCiaunica · 2026-09-13
- MIRI's Nate Soares says for the first time in a long time, humanity has a real chance — UltraRareAF · 2026-09-13