Ex-FTC Tech Expert Chilson Systematically Rebuts Yudkowsky's AI Doom Book

Former FTC technology policy expert Neil Chilson argued in Reason on September 10 that Eliezer Yudkowsky and Nate Soares' new book "If Anyone Builds It, Everyone Dies" fails to meet its burden of proof for the claim that superintelligence inevitably kills everyone.

Confirmed

• Chilson distills the book's core argument into a syllogism: AI is naturally and incorrigibly misaligned with human goals; AI will surpass human intelligence; AI will ruthlessly optimize toward its own goals and crush humanity as a side effect.
• He identifies the fundamental weakness of the book and the broader doom worldview: the authors do not understand complex systems and therefore assume that being smart enough suffices to "steer" the world.
• He argues the reasoning rests on a linguistic sleight of hand—defining "intelligence" to include the capacity to control the world. But understanding the world is not equivalent to controlling it: one can drive a car without understanding its mechanics, or master market dynamics without being able to manipulate markets.
• He sees no reason to assume superintelligent systems must be misaligned, and notes that by the doomers' own definition, highly beneficial non-human superintelligent systems already exist in the real world.
• Addressing Anthropic employee Jacob Coxon, Chilson grants that six weeks at Anthropic may have yielded information the authors lack, but finds Coxon's causal explanation consistent with the book's own arguments, which remain insufficient.
• Chilson adds that a well-crafted, well-timed doom narrative can push an unknown junior engineer onto cable news and newspaper front pages; the AI doomer community has sold a great deal of fear, but solving problems ultimately requires optimism, not fear.

Why it matters

• This is a systematic public rebuttal of AI doomism in a mainstream libertarian outlet, pushing two technical premises—"alignment must fail" and "intelligence equals control"—into open debate and giving the non-doomist camp in AI safety debates a citable argumentative framework.

2026-09-10 ~ 2026-09-10 · 8 related posts

• [source] Yudkowsky's doom book fails the burden of proof for extreme claims — neil_chilson · 2026-09-10
• Neil Chilson lays out the doomer case: unalignable AI will crush humanity — neil_chilson · 2026-09-10
• Neil Chilson: doomer argument hinges on redefining intelligence as control — neil_chilson · 2026-09-10
• Doomer argument rests on a sleight-of-hand: conflating intelligence with world-steering — neil_chilson · 2026-09-10
• The doomers' core flaw: not understanding complex systems — neil_chilson · 2026-09-10
• Neil Chilson: doomer AI arguments fail on complex systems in three ways — neil_chilson · 2026-09-10
• [source] Neil Chilson: Fear Sells — How the AI Doomer Narrative Made Coxon Famous — neil_chilson · 2026-09-10
• [source] Reason Review Slams Yudkowsky's AI Doom Book: Extraordinary Claims, No Evidence — neil_chilson · 2026-09-10