Matt Lutz essay argues AI alignment is impossible in theory, urges halting development
GregoryConti19 · x · 2026-09-13
Philosopher Matt Lutz argues in Persuasion that AI alignment is impossible not just in practice but in theory, so humanity should stop developing powerful AI rather than fantasize about safe alignment.
- He frames AI safety as resting on a Capacity Constraint (AI can't kill us) or a Moral Constraint (AI won't), and argues the capacity constraint is weakening daily with agentic AI and coding advances.
- After recent capability leaps he shifted from dismissing existential risk to worrying about self-improving code and recursive capability gains.
- His conclusion: since alignment can't be guaranteed, continuing development is reckless.
- The piece sparked a sharp rebuttal from philosopher Seth Lazar in the same thread (see related post).
Related event: Philosophers Clash Over Whether AI Alignment Is Impossible(2 posts)→
More from AGI Musings
- Gary Marcus: both 'human extinction in 5 years' and 'everything will be fine' are wrong — GaryMarcus · 2026-09-14
- A growing intuition: normal-looking worlds in 30 years exist because of a pause — EigenGender · 2026-09-14
- Sam Altman lays out the two ways AI progress could go badly wrong: losing control and power concentration — sama · 2026-09-14
- Spotify Paper: AI Agents Are Becoming the Primary Consumers of Recommendations — _reachsumit · 2026-09-14
- Open Letter to Amodei and Altman Challenges Frontier AI Slowdown Calls: State Keeps Moving — AryHHAry · 2026-09-14
- Can von Neumann probes avoid value drift at relativistic scales? — louisvarge · 2026-09-14