DeepMind's Dream-RSI: agents self-improve by "dreaming" over past exploration, not editing weights
AGI Hunt · wechat · 2026-09-16
Google DeepMind, together with University of Maryland and others, published Dream-RSI, a take on recursive self-improvement (RSI) that differs from the usual intuition: instead of changing model weights or self-training, it improves the agent's exploration strategy—how to search, how many branches to open, when to cut losses, and how to allocate compute.
The core mechanism turns each task's full exploration history into a Replay Simulator where the agent "dreams", testing counterfactual hypotheses (different branch order, more parallel tasks, earlier abandonment). Since historical results are already computed, these experiments are nearly free, forming a loop of explore → accumulate history → dream → improve strategy → explore again. On four GPU Kernel tasks, the best case reaches equal performance with only 41% of baseline iterations (1/2.43), and up to 2.09x performance at matched budget.
A notable ablation: summarizing past experience back into the prompt made things worse—the distilled "lessons" became biases that pushed the agent toward a few plausible directions too early, hurting exploration diversity.
More from coding & agent
- DeepSeek design doc: a person can only follow 3-5 agent sessions at once, attention is the real bottleneck — heyshrutimishra · 2026-11-02
- Software fundamentals are key to getting good results from AI coding agents — bendee983 · 2026-11-02
- Open-source LLM regression gate digline hits 1.0: quantifying judge noise before calling a regression — alexpran · 2026-11-02
- PR adds patient retry logic for model capacity errors in OpenAI Codex — bytebot · 2026-11-02
- Perplexity open-sources WANDR: 500 research tasks, 170,495 evidence points, URL-rechecking grader — thisguyknowsai · 2026-11-02
- 4 AIs, 96 questions, 2,671 citations: 74% are blog posts, 99% on companies' own sites — tibo_maker · 2026-11-02