Toby Ord Paper Analyzes Dynamics of Intelligence Explosions and RSI Risks
tobyordoxford · x · 2026-08-28
Toby Ord published a new paper titled "The Dynamics of Intelligence Explosions," exploring the feedback loop where AI assists in AI R&D, potentially triggering an intelligence explosion.
Key Findings & Insights:
- Difficulty of Explosive Growth: Singular growth (towards a vertical asymptote) is harder to achieve than suggested by recent economics-inspired models.
- Critical Parameter: The paper highlights "generation time" (the time to complete one feedback loop) as a pivotal but neglected parameter. Singular growth is impossible unless the generation time rapidly approaches zero.
- Safety Implications: Even without a vertical asymptote, RSI that merely doubles R&D speed (halving time for safety and governance) could quickly lead to disaster. Ord argues there is a case for requiring frontier AI companies to transparently report generation times for any AI system being improved by AI.
Paper Details:
- The 33-page paper examines growth rates that are faster than exponential but do not lead to a vertical asymptote.
- It aims to understand the most explosive possibilities to better prepare for and track them.
Link: The Dynamics of Intelligence Explosions (arXiv)
Related event: Oxford's Toby Ord Publishes "The Dynamics of Intelligence Explosions"(8 posts)→
More from AGI Musings
- AI-written cold emails hit a wall: Pragmatic Engineer's reply gets templated AI response — marktenenholtz · 2026-10-16
- UMD Philosophy hires: junior Philosophy of Mind and joint AI Ethics/Safety positions — sethlazar · 2026-10-16
- Dwarkesh: Anthropic insider expects AI to match humans at AI research within 9 months — ZeroStateReflex · 2026-10-16
- Michigan professor Ben Green: why 'AI for Good' fails and self-regulation won't work — benzevgreen · 2026-10-16
- "Everything Is an Alignment Problem": AI Models, Business Models, Teams, Providers — saranormous · 2026-10-16
- David Brin: LLMs lack 'wanting,' evolution's key product, in new machine consciousness book — cccalum · 2026-10-16