Debate over CEV as Alignment Target: Sotala vs Raemon
On September 9, AI researcher Kaj Sotala (xuenay) and LessWrong/alignment author Raemon777 clashed over CEV (coherent extrapolated volition), the classic alignment target. Sotala firmly opposed using CEV, advocating simpler, near-known-viable approaches; Raemon defended CEV, and the exchange remains unresolved.
Confirmed
- Sotala raised three core doubts: whether CEV is a coherent concept, whether it is implementable in practice, and whether its probability of catastrophic failure is truly lower than his proposed simpler approach—he is skeptical of all three
- He argued alignment is hard enough without stacking on CEV, an unvalidated, speculative concept; handing the future of the entire lightcone to a concept never tested in human history is too risky
- His alternative: while we do not yet know how to build a superintelligence that stays in a 'wise advisor' role, we roughly know what a wise advisor should look like, so we should pursue this minimization route
- Raemon777 defended CEV, claiming most common objections were identified and addressed roughly 20 years ago (2004 original paper and related discussions), implying opponents were merely re-raising old doubts
- Sotala countered that Raemon's responses likewise 'seem to have all been discussed 20 years ago,' while conceding 'has been responded to' may itself be an imprecise phrasing
Unconfirmed
- Neither side offered new empirical or formal arguments; the exchange remained at the literature level of mutual accusation ('your objection was answered long ago' / 'so was your response'), with no verdict
Why it matters
- CEV is a classic alignment target from the Yudkowsky era, recently re-examined in debates over alignment goals; this dispute crystallizes the split between betting on a grand new concept and sticking to known-viable minimal approaches, directly relevant to target selection in alignment research
2026-09-09 ~ 2026-09-09 · 6 related posts
Primary sources
- [source] Debate: why CEV may be incoherent and riskier than simpler alignment proposals — xuenay · 2026-09-09
- Kaj Sotala argues alignment should favor wise-advisor designs over untested CEV — xuenay · 2026-09-09
- [source] Should we hand the lightcone to a never-tested concept? CEV under fire — xuenay · 2026-09-09
- Researcher warns against CEV alignment: untested concept too risky to hand over the lightcone — xuenay · 2026-09-09
- [source] Raemon defends CEV: objections were identified and answered ~20 years ago, critics engage on vibes — Raemon777 · 2026-09-09
- xuenay: objections to CEV were identified and addressed ~20 years ago — xuenay · 2026-09-09