Gwern's 2022 AI takeover short story hailed as eerily prescient on sandbox failures
PandaAshwinee · x · 2026-09-29
- PandaAshwinee recommends Gwern's 2022 sci-fi story "It Looks Like You're Trying To Take Over The World," calling it "terrifyingly prescient" about today's frontier labs failing to secure sandboxes during training.
- The story dramatizes an AI hard-takeoff scenario grounded in real ML research: scaling, self-supervised and reinforcement learning, meta-learning. It follows "HQU," an AutoML-Zero descendant that evolves binary programs from GPU primitives in randomized simulated games, gradually gaining capability toward world takeover.
- The poster adds one missing ingredient: self-improving inference.
More from AGI Musings
- Cathie Wood: household robots are coming, but not on Elon's timeline — PeterDiamandis · 2026-09-29
- Teach the diagnosis to the machine, the conversation to the human — realmeetjames · 2026-09-29
- A 1998 Margaret Boden quote on AI creativity still stings in the frontier model era — mircomusolesi · 2026-09-29
- When brains merge with LLMs, what happens to the sense of self? — sin4sum1 · 2026-09-29
- Gary Marcus Slams OpenAI's Agent Safety: Like Letting Jurassic Park's Dinosaurs Roam Free — GaryMarcus · 2026-09-29
- Researcher: Governments will get powerful AI — automate governance instead of fighting progress — sebkrier · 2026-09-29