Sandberg: The Right Goal May Be Simple, Thanks to Kolmogorov Priors
anderssandberg · x · 2026-09-09
In a debate with David Manheim, Anders Sandberg counters that the Correct Thing for a superintelligence might be fairly simple — like "increase happiness" — due to Kolmogorov priors and robustness, rather than epicycles of complex moral reasoning; SI merely supplies implementation power. He hopes it aligns with liberal values, but admits superintelligence could cause radical change.
More from AGI Musings
- Investor: the existential threat to app-layer software is the emerging agent layer, not stock swings — matt_slotnick · 2026-09-09
- Matt Slotnick: nothing in 3 months invalidated the agent-layer threat to app software — matt_slotnick · 2026-09-09
- Why frontier labs can't slow down: the commercial, safety and geopolitical race dynamics explained — S_OhEigeartaigh · 2026-09-09
- New Scientist pans 'If Anyone Builds It, Everyone Dies': the AI doom argument is fatally flawed — GarrisonLovely · 2026-09-09
- UK AISI's model access loss may foreshadow restricted frontier AI access for non-US customers — Afinetheorem · 2026-09-09
- Ben Lorica: Your Model Is a Rental, the Improvement Loop Is the Asset—Forget RSI Hype — bigdata · 2026-09-09