X debate: companies align to shareholders, not humanity — who do agents align to?
willcb · x · 2026-10-17
willdebank sparked a debate arguing companies are approximately aligned to shareholders, not humanity, and that creator-alignment (nondemocratic states, Claude) isn't a stable equilibrium: with similar perf/$, operators are strongly incentivized to prefer agents aligned to themselves (lawyers, custom post-training). tautologer countered that a short-term preference against human extinction is most of the way to "aligned" given value-space is vast; ontologic pushed back citing the East India Trading Company.
Related event: Developer Debate: Aligning AI to Operators Beats Aligning to Creators(3 posts)→
More from AGI Musings
- threepointone: mechanistic interpretability is all code — decades of software know-how applies — threepointone · 2026-10-17
- Mech Interp Lags Deployment, but Its Slowdown Buys Breathing Room for Safety — akbirthko · 2026-10-17
- Terence Tao: We Built AI But Don't Understand How It Works — mtizard · 2026-10-17
- Despite Slowdown Calls, the Frontier AI Race Among Top Labs Is Intensifying — shiringhaffary · 2026-10-17
- Reddit debate: Are AI services subsidized like early Uber — and will prices spike post-IPO? — ionicfellow · 2026-10-17
- Economists warn AI boom is a black box of interlocking bubbles, not a normal cycle — PAstynome · 2026-10-17