Yudkowsky: ASI built with current techniques means ruin; P(doom) conflates two probabilities
ESYudkowsky · x · 2026-09-21
Eliezer Yudkowsky lays out why he dislikes the "P(doom)" framing: it conflates P(ruin|ASI) with P(ASI). His estimate: an ASI built by anything resembling current techniques, run by current AI company personnel, means ruin — "yes, absolutely no caveats." He argues safe ASIz outcomes exist in principle, but decades of optimists cycling through refuted hopes mean the field can't credibly commit to building only a safe ASI; ten generations of rigorous scientific correction would be needed first.
More from AGI Musings
- Naval: AI models trained on open web should be opened after ~12 months — rohanpaul_ai · 2026-09-21
- Jev and the bittersweet lesson: can differently sized models serve different apps? — amankhan · 2026-09-21
- Specialized Models Will Beat General LLMs: The Coming 'PC AI' Era — markjeffrey · 2026-09-21
- Ezra Klein's new AI video backs building mechanisms to slow AI development — NathanpmYoung · 2026-09-21
- Delta and OpenTable block AI agents, hinting at a coming platform stand-off — Scobleizer · 2026-09-21
- Stochastic parrots view 'absurd by GPT-4,' but critics say millions still haven't updated — burny_tech · 2026-09-21