Dream-RSI Lets Agents 'Dream' in Past Trajectories, Cutting Experiment Calls Up to 162x
teortaxesTex · x · 2026-09-16
teortaxesTex quotes the Dream-RSI paper's claim that "recursive self-improvement is becoming increasingly vital for autonomous AI agents," noting how quickly the Overton window has moved — RSI is now just another research topic like long context or multimodality.
The paper's idea: when real experiments are too expensive, let the agent "dream" in its own history first.
- After each real exploration, branches tried, failed approaches, evaluation results and compute costs are saved into a Discovery Tree.
- This history is reorganized into a Replay Simulator where the agent can cheaply test search strategies (which path first, when to parallelize, when to stop) without re-running real experiments.
- Better strategies go back into the real environment; new trajectories feed the simulator, forming a self-improvement loop.
- The underlying coding agent stays fixed; what evolves is the exploration-strategy layer.
Experiments span algorithm design, math optimization and GPU kernels: on Lasso tasks, agent calls drop up to 162x vs SimpleTES; on GPU kernel tasks, up to 2.09x performance gain at equal budget. History, saved completely enough, becomes a simulator for training the next generation of exploration strategies.
More from AGI Musings
- vikhyat predicts alternating compute oversupply and undersupply on the road to AGI — TheZachMueller · 2026-11-02
- Sam Altman: AI has surpassed top scientists, and 2026 will be a turning point year — kimmonismus · 2026-11-02
- DeepSeek design doc: a person can only follow 3-5 agent sessions at once, attention is the real bottleneck — heyshrutimishra · 2026-11-02
- AI researcher: we have no reliable way to tell whether an AI system is conscious — dioscuri · 2026-11-02
- Toby Ord: track both what models can and can't do to see real capability gaps — tobyordoxford · 2026-11-02
- Forward Deployed Verifier will soon be as valuable as FDE, argues poster — vaibhavbetter · 2026-11-02