Yudkowsky: ASI built with current techniques means ruin; P(doom) conflates two probabilities
ESYudkowsky · x · 2026-09-21
Eliezer Yudkowsky lays out why he dislikes the "P(doom)" framing: it conflates P(ruin|ASI) with P(ASI). His estimate: an ASI built by anything resembling current techniques, run by current AI company personnel, means ruin — "yes, absolutely no caveats." He argues safe ASIz outcomes exist in principle, but decades of optimists cycling through refuted hopes mean the field can't credibly commit to building only a safe ASI; ten generations of rigorous scientific correction would be needed first.
Related event: Yudkowsky Critiques P(doom) for Conflating Two Probabilities(2 posts)→
More from AGI Musings
- Investor: AI diffusion will be slowest in consumer software — Oura, Strava, Labcorp still have no real AI strategy — MartinGTobias · 2026-11-12
- On AI Progress: When Models Lag on Your Load-Bearing Skill, You Can't See Them Improving — dfrsrchtwts · 2026-11-12
- Slack's David Hoang: agents beat people on coordination, proposes 'boundary agents' — davidhoang · 2026-11-12
- CS student mourns coding flow states he'll never experience in the AI era — chrissyykat · 2026-11-12
- Ford's 370% Turnover Led to $5 Days — But ASI Never Quits — krishnan · 2026-11-12
- Against 'verifiable vs. fuzzy': the real axis is how expensive tasks are to grade — dfrsrchtwts · 2026-11-12