Kaj Sotala: CEV is a wish list, not a design — alignment lacks mechanisms
xuenay · x · 2026-09-09
In a debate with Raemon, Kaj Sotala argues CEV (Coherent Extrapolated Volition) isn't a "design" like a nuclear reactor blueprint — it's a list of desiderata with no mechanistic story for achieving them, equivalent to "we'd like a reactor that doesn't blow up." He says he can't offer detailed criticism because so little exists to criticize: opponents cite the intent, but only a wish that it would work actually exists.
Related event: Alignment Researchers Debate Whether CEV Is a Design or a Wish List(3 posts)→
More from AGI Musings
- Nonfiction book market is collapsing, and authors are memeing about it on X — jjvincent · 2026-09-09
- Former Mosaic researcher mocks the "user data flywheel" moat narrative in viral thread — bookwormengr · 2026-09-09
- FT's Burn-Murdoch: ChatGPT-assisted coursework means schools are no longer assessing kids at all — jburnmurdoch · 2026-09-09
- Anthropic researcher: novelty-rewarded RL may teach agents to obfuscate their sources — suchenzang · 2026-09-09
- "A country of geniuses in a datacenter": Tuvalu-scale AGI quip — nabla_theta · 2026-09-09
- LWiAI Podcast #256: Fable 5.1 Price Cuts, Astra Zero-Day Claims, and New Details on the OpenAI-Hugging Face Incident — Last Week in AI · 2026-09-09