New LessWrong essay on "the self-unalignment problem" in AI alignment
jankulveit · x · 2026-10-27
Jan Kulveit and Rose Hadshar published "The self-unalignment problem" on LessWrong, a theory piece on a novel failure mode in alignment: a system (or its subagents) drifting away from the values it was supposed to extrapolate, rather than merely diverging from human intent.
The post sits within the Coherent Extrapolated Volition, failure mode, subagents, and value learning threads of alignment research, and has scored 160 points on the site.
More from Safety
- Negative result: ignorance-based tamper-resistance only works for broad knowledge domains — StephenLCasper · 2026-10-28
- CATCH: ~200 examples of adversarial training make LLMs robustly honest across benchmarks — irinarish · 2026-10-28
- Anthropic disclosure: no insurer, or even the whole industry, could cover catastrophic AI risk — wfithian · 2026-10-28
- Tonawanda Seneca Nation opposes data center on wetland at People's AI Assembly — AINowInstitute · 2026-10-28
- Cloudflare launches PHP CTF bug bounty challenge to stress-test its WAF — dinasaur_404 · 2026-10-28
- agent-repo-preflight: Open-Source Read-Only Audit of AGENTS.md, MCP and Hook Configs Before Cloned Repos Hit Your Coding Agent — Digital-Fortress-Sec · 2026-10-28