The Famous '10% AI Extinction Risk' Number Is Being Epistemically Laundered, Critics Argue
sebkrier · x · 2026-09-18
A debate over the oft-cited AI extinction number: Luke Muehlhauser amplified a claim that several top mathematicians (including multiple Fields medalists) consider the ≥10% chance of human extinction from AI within a decade to be a genuine emergency, not hype.
Itai Sher pushed back on the epistemics: the 10% figure was originally floated by Anthropic employees without public justification, then picked up by mathematicians — and is now attributed to their independent judgment. He calls this a strange laundering of the number: surveying AI practitioners is fine, but posts like this are bad epistemic practice.
More from AGI Musings
- Researchers argue p(doom) launders evidence-free fears through pseudo-quantification — RexDouglass · 2026-09-18
- Researcher chose genetics over AI in 2020, citing an unimaginable superintelligence future — erikphoel · 2026-09-18
- How Yudkowsky and Bostrom shaped today's AI safety discourse 12-16 years ago — Justin_Halford_ · 2026-09-18
- The intelligence loan: frontier labs subsidize your access and repay themselves by distilling your edge — cjmaddison · 2026-09-18
- The knowledge economy is over: Dan Shipper on the 'Allocation Economy' where every maker becomes a manager — danshipper · 2026-09-18
- The 'Claude Coworker' does all his work unchecked—and the AI now outperforms him — omooretweets · 2026-09-18