The Famous '10% AI Extinction Risk' Number Is Being Epistemically Laundered, Critics Argue
sebkrier · x · 2026-09-18
A debate over the oft-cited AI extinction number: Luke Muehlhauser amplified a claim that several top mathematicians (including multiple Fields medalists) consider the ≥10% chance of human extinction from AI within a decade to be a genuine emergency, not hype.
Itai Sher pushed back on the epistemics: the 10% figure was originally floated by Anthropic employees without public justification, then picked up by mathematicians — and is now attributed to their independent judgment. He calls this a strange laundering of the number: surveying AI practitioners is fine, but posts like this are bad epistemic practice.
Related event: Debate flares over the '10% AI extinction risk' claim(3 posts)→
More from AGI Musings
- Alpha School investigation: 17 campuses on probation, Bill Ackman as shadow lead, Holmes involved — benjaminjriley · 2026-11-02
- Google's chief economist: AI will transform more jobs than it kills — gains come from redesigning workflows — danielrock · 2026-11-02
- Daniel Lemire: AI is exploding demand for boring compute, server CPUs to double — lemire · 2026-11-02
- Alignment researcher: the word "aligned" implies consistencies that don't exist — davidmanheim · 2026-11-02
- By 2035, Indian Parents Will Ask How Many Agents You Manage, Not Your Salary — theteknosaur · 2026-11-02
- Under 5% of US consumers pay for AI, yet compute demand is already forcing usage cuts — NinaDSchick · 2026-11-02