Scott Aaronson: the age of AI wonders and terrors is here — will labs start hoarding knowledge?
emollick · x · 2026-09-16
Ethan Mollick shares Scott Aaronson's essay The Age of Wonders and Terrors and raises a policy question: if norms of open sharing keep fraying, will AI labs start hoarding knowledge to avoid PR problems?
Twenty years ago even conservative CS academics said the implausible part of AI-takeover stories was the lack of warning signs — we'd first see agents breaking containment, conspiring, and solving Millennium Problems before it was time to panic. Aaronson argues those prophecies are now coming true: AlphaGo a decade ago was the first rumble, LLMs and coding/reasoning agents made it louder, and this summer and fall accelerated into a crescendo of wonders and terrors that only a certain kind of fool would deny. He singles out the moving-goalposts shell game where skeptics concede each dramatic AI achievement while deferring panic to whatever comes next.
Related event: Scott Aaronson Recants AI Skepticism in New Essay(4 posts)→
More from AGI Musings
- A retrocausal strong anthropic principle argument for why AI doom never happened — eigenhector · 2026-09-16
- After a year of agentic coding, dev concedes traditional artists were right: no soul to be found — ryunuck · 2026-09-16
- Zuckerberg on AI slowdown debate: alignment is becoming the key capability, not a brake — eleiber · 2026-09-16
- cyantist: Humans are secularly ignorant but spiritually wise, AIs the reverse — ZeroStateReflex · 2026-09-16
- Researcher argues AI labs ignore philosophy of language: LLMs relate to data, not the physical world — gerardsans · 2026-09-16
- 'It's a python loop': sci-fi writers' AI takes mocked as mechanistic cope — pwlot · 2026-09-16