Researcher warns against CEV alignment: untested concept too risky to hand over the lightcone
xuenay · x · 2026-09-09
AI researcher xuenay argued against using CEV (Coherent Extrapolated Volition) as an alignment target in a debate with Raemon777.
- Alignment is hard enough without stacking an even more speculative, never-validated concept on top
- Better path: build a wise advisor to consult with, figure out whether CEV even makes sense, then consider implementing it
- He asks why we would turn over the lightcone to something "literally never tested in the history of humanity"
Related event: Alignment Researchers Push Back on CEV as Unproven and Risky(4 posts)→
More from AGI Musings
- kalomaze: labs mine user data for correlated novel failure classes, not one-off edge cases — kalomaze · 2026-09-09
- Anything verifiable is extremely soluble for AI — it's just a matter of time — vxnuaj · 2026-09-09
- kalomaze: frontier training gains come from domain-level signals, not power users — kalomaze · 2026-09-09
- AI Optimism essay: AI is easier to control than human labor — a technical case for alignment — QuintinPope5 · 2026-09-09
- AI lab's Millennium problem run burned 300B output tokens, $20-30M at consumer prices — Paimaamu · 2026-09-09
- Zachary Lipton: Author credit lasted 3,000 years — how long will prompter credit last? — zacharylipton · 2026-09-09