Kaj Sotala argues alignment should favor wise-advisor designs over untested CEV
xuenay · x · 2026-09-09
In a debate with Raemon about alignment approaches, researcher Kaj Sotala lays out his stance: be careful and stick to minimal solutions close to what's known to work. He argues we don't know how to build a superintelligence that would stick to a wise advisor role, but we do roughly know what wise advisors are and should be like—making it a safer baseline than speculative schemes.
Related event: Alignment Researchers Push Back on CEV as Unproven and Risky(4 posts)→
More from AGI Musings
- Scientific American: AI may have solved a million-dollar math problem, a Deep Blue moment — MelMitchell1 · 2026-09-09
- Melanie Mitchell: Deep Blue never became AGI, and neither will this math breakthrough — MelMitchell1 · 2026-09-09
- LEAP expert panel puts just 10% odds on AI cracking a Millennium Problem by 2027 — Afinetheorem · 2026-09-09
- LEAP panel of AI experts and superforecasters gives just 10% odds of AI solving a Millennium Problem by 2027 — Afinetheorem · 2026-09-09
- American Mathematical Society Weighs In: Navier-Stokes Progress Is a Milestone — AlexKontorovich · 2026-09-09
- Why 1k-10k agents per strong math solution may be the right order of magnitude — cephaloform · 2026-09-09