Neil Chilson lays out the doomer case: unalignable AI will crush humanity
neil_chilson · x · 2026-09-10
In his running critique, Neil Chilson summarizes the core argument of Yudkowsky and So8res's book: AI is inherently unaligned (and unalignable) with human goals, will surpass human intelligence, and will relentlessly optimize toward those goals, crushing humans as a side effect. He also pushes back on claims that inside Anthropic knowledge supports the thesis, arguing such extreme claims carry a high burden of proof that the book's arguments fail to meet.
More from AGI Musings
- Is quantifying uncertainty arbitrary? A Bayesian ideology debate — NathanpmYoung · 2026-09-11
- Zvi disputes claim that AI lost a live debate against conservatives in Dallas — TheZvi · 2026-09-11
- Matt Turck talks with Richard Socher on recursive self-improvement and AI-driven science — mattturck · 2026-09-11
- OpenAI Foundation commits $60M to bring AI weather forecasts to 100M smallholder farmers — jachiam0 · 2026-09-11
- Researchers draft reviewer statement demanding venues verify author involvement — JessicaHullman · 2026-09-11
- Wolf: Alignment Won't Be Solved Behind Closed Doors at Frontier Labs — Thom_Wolf · 2026-09-11