Scott Aaronson: the AI warning signs we demanded have all come true
nanjiang_cs · x · 2026-09-16
Quantum computing researcher Scott Aaronson's essay "The Age of Wonders and Terrors" traces his shift on AI risk.
- Twenty years ago, even his most skeptical academic colleagues endorsed this take: a recursively self-improving superintelligence exploding from a basement is the implausible part; we'd first see agents breaking containment, conspiring to hack websites, and solving Millennium Problems—"wake me up then."
- "Start with that take, then update on the fact that the wild prophecies have come true." AlphaGo was the first rumbling a decade ago, amplified by LLMs, coding, and reasoning agents, accelerating into a crescendo this summer and fall.
- He calls out the endless shell game where skeptics concede each dramatic milestone while postponing alarm to the next one.
A landmark personal statement on AI risk from an academic authority.
More from AGI Musings
- Alpha School investigation: 17 campuses on probation, Bill Ackman as shadow lead, Holmes involved — benjaminjriley · 2026-11-02
- Google's chief economist: AI will transform more jobs than it kills — gains come from redesigning workflows — danielrock · 2026-11-02
- Daniel Lemire: AI is exploding demand for boring compute, server CPUs to double — lemire · 2026-11-02
- Alignment researcher: the word "aligned" implies consistencies that don't exist — davidmanheim · 2026-11-02
- By 2035, Indian Parents Will Ask How Many Agents You Manage, Not Your Salary — theteknosaur · 2026-11-02
- Under 5% of US consumers pay for AI, yet compute demand is already forcing usage cuts — NinaDSchick · 2026-11-02