Anthropic researchers go public on extinction risk as deployers face very different worries
Dapper-Tale-4021 · reddit · 2026-09-12
Jacob Coxon resigned from Anthropic this week specifically to say publicly that both OpenAI and Anthropic are "gambling with our lives" racing toward self-improving superintelligence. Alignment science lead Evan Hubinger confirmed: "Jacob is correct here, we really do earnestly believe AI could kill all humans," putting it above 10% within the decade and admitting Anthropic has no plan for aligning superintelligence. Scalable oversight lead Samuel Marks said similar — the safety team at the safety-focused lab publicly agreeing with the person who quit over safety.
The author rejects both the "marketing hype" and "genuine terror" reads, and highlights a practical disconnect: in enterprise deployment rooms nobody discusses extinction — they worry about an agent with CRM write access doing something stupid at 3am and who signs off when output is wrong. If the builders can't agree whether it's an existential threat, what should a mid-sized company base its risk assessment on?
Related event: Insiders at Top AI Labs Warn of Extinction Risk; Musk Calls It "Psyops"(17 posts)→
More from AGI Musings
- Tao and Fields Medalists' two objections to AI in math, and why they're weak — RexDouglass · 2026-09-12
- Balaji: Courage and agency are the last moat in the AI era — beffjezos · 2026-09-12
- jon_stokes argues Yudkowsky's definition of intelligence is narrow and brittle — nptacek · 2026-09-12
- Physicist mourns as AI compresses the timelines of what's possible in math — kylekabasares · 2026-09-12
- Mathematicians' meltdown over AI cracking Navier-Stokes mirrors a real Atlantic piece: 'I'd Rather Risk Cancer Than See AI Move This Fast' — adam_dorr · 2026-09-12
- 15-year consultant: no one 'picks things up' across industries — binarybits · 2026-09-12