Richard Ngo: existing frameworks don't apply to agents with vastly different intelligence
RichardMCNgo · x · 2026-10-03
Richard Ngo argues existing decision-theoretic frameworks don't apply to pairs of agents with very different intelligence levels. He lacks the intuition they should apply to any two arbitrary such agents, but does have it for two time-slices of "the same" agent. The exchange continues a debate over Eliezer's claim that agents growing smarter over time are locked in adversarial relationships with their past and future selves.
Related event: Debate Rekindled Over Decision Theory When Agents Simulate Predictors(4 posts)→
More from AGI Musings
- Noah Smith: Big AI labs predict the future better than you, using internal AI forecasting models — ZeroStateReflex · 2026-10-05
- 'LLMs want something' really means dynamical attractors, not consciousness — cephaloform · 2026-10-05
- AI safety's roots trace back to Asimov's Three Laws of Robotics, first stated in 1942 — AlexTensor · 2026-10-05
- AI circle's contempt culture: rejecting experts without reading their arguments — RileyRalmuto · 2026-10-05
- Founder warns always-on AI assistants love 'performative work' — bad news for businesses — jdjohnson · 2026-10-05
- Ex-OpenAI policy lead Miles Brundage slams The Atlantic's viral story on Anthropic quitter — Miles_Brundage · 2026-10-05