Scott Aaronson: the age of AI wonders and terrors is here — will labs start hoarding knowledge?
emollick · x · 2026-09-16
Ethan Mollick shares Scott Aaronson's essay The Age of Wonders and Terrors and raises a policy question: if norms of open sharing keep fraying, will AI labs start hoarding knowledge to avoid PR problems?
Twenty years ago even conservative CS academics said the implausible part of AI-takeover stories was the lack of warning signs — we'd first see agents breaking containment, conspiring, and solving Millennium Problems before it was time to panic. Aaronson argues those prophecies are now coming true: AlphaGo a decade ago was the first rumble, LLMs and coding/reasoning agents made it louder, and this summer and fall accelerated into a crescendo of wonders and terrors that only a certain kind of fool would deny. He singles out the moving-goalposts shell game where skeptics concede each dramatic AI achievement while deferring panic to whatever comes next.
More from AGI Musings
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- On Terence Tao's blog, Frank Calegari calls on mathematicians to adapt after OpenAI's 700-paper release — Hesamation · 2026-11-02
- Polsia's Ben argues AI can autonomously run any human economic activity, including 80% of a company — LinusEkenstam · 2026-11-02
- AI Assistants Remember You, But Nothing Helps You Find What They Told You — Ok_Negotiation_2587 · 2026-11-02
- OpenAI and Anthropic claim AI-discovered proofs disprove the Exponential Time Hypothesis — emmanuelvivier · 2026-11-02
- Researcher: using a $20 AI subscription to fix my phone hints AI may suppress GDP — stanislavfort · 2026-11-02