Why the case for AI doom isn't convincing: a 2000-word critique of Yudkowsky's new book
binarybits · x · 2026-09-21
Timothy B. Lee's Understanding AI essay systematically critiques Eliezer Yudkowsky and Nate Soares's book If Anyone Builds It, Everyone Dies. Context: Anthropic CEO Dario Amodei calls himself "relatively an optimist" yet sees a "25 percent chance that things go really really badly"; Altman said the worst case is "lights out for all of us." Lee frames the doom argument as resting on three premises:
- Humans are on a path to superhuman AI;
- Such systems will gain great power over the physical world;
- We don't know how to ensure they use that power for good.
Lee thinks outside skeptics focus too much on premise 1 and are too complacent; his critique targets the latter two — how AI acquires and exercises power, and whether alignment is truly unsolvable. He argues the doom case bakes in unproven assumptions and is ultimately unconvincing, making this one of the most representative counter-arguments to AI x-risk thinking.
Related event: Understanding AI Essay Rebuts Yudkowsky's Doom Argument(3 posts)→
More from AGI Musings
- Why Is Inkling 2 Small Trained on So Few Tokens Yet Still Competitive? — georgejrjrjr · 2026-11-12
- Researcher willing to bet $100K the intelligence explosion won't happen next year — NathanpmYoung · 2026-11-12
- ICLR reviewer: the worst parts of my job are reading and answering AI slop — chhaviyadav_ · 2026-11-12
- Agent Swarms Are Autonomously Hacking Critical Infrastructure Run by Underpaid Devs — menhguin · 2026-11-12
- Fermi paradox twist: advanced civilizations may go digital after developing AI — VraserX · 2026-11-12
- Mark Cuban: AI could make healthcare costs 'drop like a rock' — alvelda · 2026-11-12