New essay dissects recursive self-improvement: the anatomy of an intelligence explosion
sebkrier · x · 2026-11-16
Aashish Reddy's Cosmos Institute essay "Recursive, Self, Improvement" is being shared as a rigorous philosophical treatment of RSI, beyond just a compute-scaling curve.
- Opens from I.J. Good's 1965 intelligence explosion argument: the first ultraintelligent machine may be humanity's last invention.
- Observes that AI now helps AI researchers write code and run experiments, growing more autonomous — "AIs are training the next AIs" — and that researchers, policymakers, and lab executives feel RSI is here or imminent.
- Core thesis: most progress comes from object-level improvements; the Scientific Revolution was a discontinuity via improved metaknowledge, but RSI is more discontinuous because intelligence turns its problem-solving abilities onto those very abilities.
- Builds on Yudkowsky's framing to explore implications for safety and control, where steering RSI accumulates within whichever organization leads it.
Related event: Cosmos Institute Essay Dissects Recursive Self-Improvement(2 posts)→
More from AGI Musings
- Want human-level AI? Don't work on LLMs. Want superhuman AI? Do — repligate · 2026-11-16
- Gary Marcus asks: Is GTA 6 more impressive than GPT 6? — GaryMarcus · 2026-11-16
- "AI should be the greatest redistribution of power in history, not concentration" — natesiggard · 2026-11-16
- "Programmers are now a liability": solo founder keeps only coding outsourced — yangyi · 2026-11-16
- Epic CEO Tim Sweeney predicts AI could bring 10X productivity and new game genres — draginol · 2026-11-16
- AI 'magic' runs on expert judgment: data labeling now means capturing professional know-how — sebpaquet · 2026-11-16