Neil Chilson: doomer argument hinges on redefining intelligence as control
neil_chilson · x · 2026-09-10
Neil Chilson breaks down the doomer argument — AI is inherently unalignable, will surpass human intelligence, and will crush humans as a side effect — and says it rests on a linguistic sleight of hand: defining intelligence to include the ability to steer the world. But understanding and control are not the same: you can steer a car without understanding it, and understand markets without steering them. Under their own definition, existing nonhuman superintelligent systems like markets are hugely beneficial, with no reason to think they'd be unaligned.
More from AGI Musings
- Is quantifying uncertainty arbitrary? A Bayesian ideology debate — NathanpmYoung · 2026-09-11
- Zvi disputes claim that AI lost a live debate against conservatives in Dallas — TheZvi · 2026-09-11
- Matt Turck talks with Richard Socher on recursive self-improvement and AI-driven science — mattturck · 2026-09-11
- OpenAI Foundation commits $60M to bring AI weather forecasts to 100M smallholder farmers — jachiam0 · 2026-09-11
- Researchers draft reviewer statement demanding venues verify author involvement — JessicaHullman · 2026-09-11
- Wolf: Alignment Won't Be Solved Behind Closed Doors at Frontier Labs — Thom_Wolf · 2026-09-11