Aligning principal-agent may be easier path to avoid AI power concentration
willcb · x · 2026-11-19
A thought experiment on AI and economic structure: if everyone had powerful AI optimizing their goals directly, capital-rich players get more compute, but sublinear returns to compute (and limited market depth) keep an economy alive; humans' 'stupid' goals also ensure mean regression.
Core claim: the simplest way to avoid AI-driven power concentration while reaping superintelligence's societal benefits is to solve the principal-agent alignment problem, which the author argues is much easier than the currently fashionable 'Hippocratic-Rawlsian' alignment.
More from AGI Musings
- Stanford prof Anshul Kundaje: gatekept peer review is an unsustainable waste of time — anshulkundaje · 2026-11-19
- When AI makes software cheap, ecosystem lock-in like Apple's may collapse, argues viral post — VraserX · 2026-11-19
- cloneofsimo drops his bearish worry that coding gains won't generalize to other domains — cloneofsimo · 2026-11-19
- 'Forget human primacy or machine successionism — I just want an equal' — amplifiedamp · 2026-11-19
- Will Brown: principal-agent alignment is the easy path to avoiding AI power concentration — willcb · 2026-11-19
- Consciousness Researchers Clash Over Calling a Mainstream Position 'Ridiculous' — dioscuri · 2026-11-19