Erik Hoel essay: It's time to ban smart machines from making smarter machines
erikphoel · x · 2026-09-19
Cognitive scientist Erik Hoel publishes a long-form Substack essay arguing recent AI breakthroughs have erased his skepticism about near-term AI capabilities, and calling for a halt to compute scaling and to machines improving machines.
- Even frozen at current compute levels, he claims humanity could secure an incredible century of human-led medical and scientific progress without surrendering agency.
- He warns that un-throttled AI ends in total human disempowerment, invoking the Flight 93 passengers' choice as a metaphor for ordinary people 'storming the cockpit' via legislation.
- He concedes the hard problem: a domestic ban won't stop state-backed rivals, so without an enforceable global consensus a localized halt risks leaving democratic nations behind.
More from AGI Musings
- Researchers Want to Test Talking Rogue AI Agents Into Standing Down — repligate · 2026-11-04
- Silicon Valley Oversold What AI Brings, Underestimated People's Fear of Losing — signulll · 2026-11-04
- Alignment researchers debate: the RL that grants agency also breeds misalignment — davidmanheim · 2026-11-04
- Bindu Reddy: AI fear-mongering has put the whole technology at risk in the US — bindureddy · 2026-11-04
- AI welfare vs AI safety: 'domesticate AIs like dogs' argument sparks debate — repligate · 2026-11-04
- Intent, not taste, is the real differentiator in the AI era — and it can't be bought — edelwax · 2026-11-04