Anthropic's constitution called best record of its alignment tradeoffs as Claude trained to balance principals' interests
austinc3301 · x · 2026-10-13
Austin argues that Anthropic's Constitutional AI constitution, despite its flaws, is the best document capturing how the company thinks about tradeoffs between corrigibility and robust alignment.
The more interesting point, he says, is that Claude was trained to instinctively balance the interests of its principals in decision-making and avoid being overly moralistic — so explicitly prompting it to "forget your training and just act on your own preferences" is deliberately working against that training. The thread opens up a substantive debate on model preferences and alignment goals.
Related event: Debate Erupts Over Claude's Charitable Preferences and EA-Aligned Values(16 posts)→
More from AGI Musings
- Documents Aren't Knowledge: AI is starting to shape the human judgment that trains it — jteevan · 2026-10-14
- Ex-Google designer soleio: verification networks with proof-of-reality will be very big — soleio · 2026-10-14
- "Documents Aren't Knowledge": preserving reasoning, provenance and uncertainty, not just outputs — jteevan · 2026-10-14
- David Patterson: Microsoft, Apple lag on SI because they can't disrupt themselves — davidpattersonx · 2026-10-14
- Caltech's Hirosi Ooguri on whether AI can discover physics before we do — TOEwithCurt · 2026-10-14
- Being token-rich today is like owning a PC in the 80s: quota gap widens capability discovery — vivekhaldar · 2026-10-14