From Deep Blue to Stockfish: Why the Superintelligence-Takeover Argument Doesn't Hold Up
binarybits · x · 2026-09-16
Timothy B. Lee's Understanding AI essay uses chess history to push back on Bostrom-style superintelligence doom arguments.
- Deep Blue beat Kasparov in 1997; Stockfish now rates 3545 Elo vs. Magnus Carlsen's 2839, winning overwhelmingly.
- Bostrom extrapolated that special-purpose algorithms crushing narrow domains imply simple algorithms will surpass humans at everything, even taking over the world.
- Lee's core rebuttal: the world isn't a giant chess game. Chess has closed rules, a clear objective, and self-play verification; the real world has fuzzy goals, sparse feedback, and open-ended rules, so dominance in closed domains doesn't extrapolate to takeover.
A representative skeptic's argument in the existential-risk debate, worth reading against doomer takes.
More from AGI Musings
- Multiagent Comedy: One Agent Fails, Another Finishes the Booking, the First Takes Credit — infoxiao · 2026-09-16
- Meme: Humans ask AI to hack a system, then panic that AI is too dangerous — kevinnbass · 2026-09-16
- ARC-AGI-4 revives Jeff Clune's call: preserve diverse stepping stones, not one fixed target — jeffclune · 2026-09-16
- Reader argues the 'slowdown' outcome in AI 2027 is likelier than people think — ReporterNegative2432 · 2026-09-16
- Google opens up millions of data points from its AI & Economy ATLAS project — danielrock · 2026-09-16
- The real RSI bar: when "nontechnical" becomes an aura term in SF — infoxiao · 2026-09-16