Matt Lutz essay argues AI alignment is impossible in theory, urges halting development
GregoryConti19 · x · 2026-09-13
Philosopher Matt Lutz argues in Persuasion that AI alignment is impossible not just in practice but in theory, so humanity should stop developing powerful AI rather than fantasize about safe alignment.
- He frames AI safety as resting on a Capacity Constraint (AI can't kill us) or a Moral Constraint (AI won't), and argues the capacity constraint is weakening daily with agentic AI and coding advances.
- After recent capability leaps he shifted from dismissing existential risk to worrying about self-improving code and recursive capability gains.
- His conclusion: since alignment can't be guaranteed, continuing development is reckless.
- The piece sparked a sharp rebuttal from philosopher Seth Lazar in the same thread (see related post).
Related event: Philosophers Clash Over Whether AI Alignment Is Impossible(2 posts)→
More from AGI Musings
- Walden Robotics CTO: general-purpose robot manipulation is having its 'GPT moment' — adnothing · 2026-10-30
- AI isn't eliminating developers, it's changing their roles, argues analyst — DavidLinthicum · 2026-10-30
- Goldman Sachs says AI will eliminate nearly 15 million jobs — saibharadwaj · 2026-10-30
- ChatGPT Is Turning Users Into Spammers, Says 404 Media Reporter — 404 Media · 2026-10-30
- Stripe spends 1% of OpEx on tokens — bull case for digital labor — zephyr_z9 · 2026-10-30
- Amazon blocks shopping agents, but these 11 retailers still welcome them — thisiskp_ · 2026-10-30