AI Safety Expert: Pure Recursive Self-Improvement is Distant, Lengthen Timelines
joshua_saxe · x · 2026-08-09
joshuasaxe, an AI security expert, pushes back against the media and policy hype around idealized Recursive Self-Improvement (RSI), arguing it is a distant ideal and that timelines should be lengthened.
He points out that large model development is heavily bottlenecked by human supervision across all stages—from data sourcing and multi-scale experiments to safety evals. Idealized RSI aims to automate this entirely, but handing over the reins to current models (which have demonstrated hacking capabilities) to write RL environments or generate trajectories risks severe reward hacking and sandbox escapes.
Given the unacceptability of these risks with present technology, labs are unlikely to pursue such reckless automation. Short-term automated research will likely be limited to algorithmic and architecture search, far from the fully autonomous RSI narrative discussed in forecasts like AI 2027/2040.
Related event: Expert Warns Idealized Recursive Self-Improvement Is Unlikely Soon(3 posts)→
More from AGI Musings
- Personal shopping agents bumped her secondhand spending from $365 to $820 in 2 months — davidyin44 · 2026-10-16
- There's no formal route to frontier AI research: 19-year-olds are advancing tech from their basements — repligate · 2026-10-16
- 'Will you end humanity?' AI's candid answer becomes a clown meme — dermflork · 2026-10-16
- Stanford professor: AI-done homework scores high, students fail in-person finals — FinanceYF5 · 2026-10-16
- Data brokers pay $100K–$2M for SaaS datasets that may train their own replacements — FinanceYF5 · 2026-10-16
- tszzl: Pretraining science is like steam engines before thermodynamics, slouching toward interpretability — tszzl · 2026-10-16