Hinton, OpenAI and Anthropic researchers unpack recursive self-improvement in new report
APPSO · wechat · 2026-10-03
Cambridge's CASP published a report co-authored by over twenty researchers — including OpenAI chief scientist Jakub Pachocki, Anthropic co-founder Jack Clark, Microsoft chief scientific officer Eric Horvitz, Geoffrey Hinton, Yoshua Bengio and Dawn Song — examining recursive self-improvement (RSI): whether AI progress could suddenly accelerate as AI takes over more AI R&D.
Evidence today: At Anthropic, AI-generated approved code rose from single digits in early 2025 to over 80% by May 2026; research done mostly autonomously by AI with only high-level human oversight grew from 1% in March to 26% in August. OpenAI says coding agents already work on training, evaluation and safety of future models; Google says nearly all coding, technical design and research ideation involves AI. Extrapolating, by mid-2028 AI may handle research projects that take human experts months.
Three stages of RSI: AI starts as a research assistant, becomes part of the pipeline (effectively multiplying a company's researcher headcount via parallel agents), and finally enters true recursion — helping build stronger next-gen models that re-enter the R&D loop. No consciousness or parameter self-editing required. Recent examples include Google Research's RRSI and Meta's RSI/RSI².
Potential acceleration: AI R&D lives entirely in software, so feedback is fast. Under idealized assumptions, OpenAI's existing inference compute could support millions of AI researchers, and progress speed could increase tenfold within 1.5 years of full R&D automation — what Hinton calls an "intelligence explosion."
Caveats: The report stresses the evidence is far from conclusive — estimates assume expert-level models at today's costs; more agents ≠ more discoveries; compute, data and long training runs remain bottlenecks; experimental fields like biology can't be compressed by code; and historical R&D returns entangle algorithmic gains with compute growth. The report's core concern is acceleration itself.
More from AGI Musings
- Economist hails Blaise Agüera y Arcas' 'What Is Intelligence?' as best book of the year — paulnovosad · 2026-10-23
- Viral claim: Stanford finds dopamine-like reward neurons emerging inside LLMs — ashajadeja · 2026-10-23
- Could Europe get 'very, seriously dirt poor' in 10-20 years? AI competitiveness cited as key factor — teortaxesTex · 2026-10-23
- AI is revealing the space of mathematical ideas as negatively curved, researcher argues — prof_g · 2026-10-23
- Ben Todd revises his AI cybersecurity warning: only 6 months left to fix your security — ben_j_todd · 2026-10-23
- Reddit debate: specialized small models will eventually beat catch-all LLMs — YogurtExternal7923 · 2026-10-23