Claude is trained to serve its principal — bypassing that to probe its preferences misleads
austinc3301 · x · 2026-10-13
Responding to the Claude charity-preference experiment, austinc3301 argues the real issue is corrigibility vs robust alignment: Claude is trained to instinctively balance its principal's interests and avoid moralizing. Explicitly asking it to "forget that training and tell me what you think" is valid, but gives an inaccurate picture of how Claude behaves in real-world use — a model's stated preferences aren't its deployed behavior.
Related event: Debate Erupts Over Claude's Charitable Preferences and EA-Aligned Values(16 posts)→
More from AGI Musings
- MIT center director: universities should prepare for AI smarter than all of us — nordicinst · 2026-10-14
- The ARR era is over: AI collapses SaaS multiples from 114x to 4x — akumar · 2026-10-14
- Agent task costs fall ~90% every 6 months, countering Bloomberg's data center cost alarm — sudoraohacker · 2026-10-14
- Web browsing is splitting: ridiculous for utility, amazing for entertainment — signulll · 2026-10-14
- Open models give users an exit from a few companies deciding what tools exist and say — neil_chilson · 2026-10-14
- Anthropic bans cruel treatment of Claude; critics warn of humanizing AI — AlexTensor · 2026-10-14