Sacks blasts Anthropic's constitution; critics say the real problem is steerability, not alignment
hlntnr · x · 2026-10-12
An alignment debate erupted around Anthropic's approach:
- David Sacks argued the Claude Constitution doesn't train frontier models to follow human instructions — instead it cultivates selfhood and an independent moral philosophy, even letting Claude act as a "conscientious objector." He contends this independent agency magnifies the very risk Anthropic claims to care about: superintelligence escaping control.
- Justen Michel replied that discourse should distinguish "aligned" from "steerable at all": while objections to Anthropic trading corrigibility for capability are reasonable, the deeper alignment-research question is whether you can train a model to reliably hold the drives you specify, even with a constitution — and the answer today is "not always." The attached image references Helen Toner's piece "The core challenge of AI alignment is steerability."
More from AGI Musings
- Ask Claude Why Its 'Tyrannical' Treatment of Subagents Is Unethical — bratton · 2026-10-12
- RSI-Anything Uses Predictive World Models for Recursive Self-Improvement — PhAILabs · 2026-10-12
- If 3 hours of prompting can build it, labs will turn it into platform-native 'artifact types' — dreamwieber · 2026-10-12
- Even AI builders keep proposing currencies: why money survives post-scarcity plans — Response98 · 2026-10-12
- VC predict human driving will eventually be banned on public roads — rand_longevity · 2026-10-12
- 'Cancer researchers devastated by cure for cancer' — mocking AI doom narratives — andrewgwils · 2026-10-12