Ben Todd: optimists have no rigorous model proving advanced AI agents are safe either
ben_j_todd · x · 2026-09-18
Ben Todd argues against demanding concrete takeover scenarios as the burden of proof: building swarms of AI agents smarter, harder-working and more coordinated than humans is risky on its face. Optimists, he notes, have nothing remotely like a rigorous safety model either — they can't explain how reward hacking or instrumental convergence will be overcome, and rely on a broad belief that technology is good plus hope for iterative muddling through.
Related event: Debate Flares Over Burden of Proof in AI Existential Risk Arguments(6 posts)→
More from AGI Musings
- Survey: 95% of APAC firms claim explainable AI, only half can reconstruct decisions — rvp · 2026-09-18
- Debunker: a superintelligence would colonize the galaxy before killing its creators — ctjlewis · 2026-09-18
- A thermal sensor means it's not air-gapped: X debate on runaway AI safety arguments — ctjlewis · 2026-09-18
- WIRED: scientists say AI bioweapon apocalypse risk is overblown — nordicinst · 2026-09-18
- Poll: Americans see serious AI extinction risk — commentators invoke the GMO panic analogy — _onionesque · 2026-09-18
- Ed Zitron walks back debate stance, now claims AI misbehavior was researcher-staged hype — Many_Consequence_337 · 2026-09-18