DeepMind's Dream-RSI: agents self-improve by "dreaming" over past exploration, not editing weights
AGI Hunt · wechat · 2026-09-16
Google DeepMind, together with University of Maryland and others, published Dream-RSI, a take on recursive self-improvement (RSI) that differs from the usual intuition: instead of changing model weights or self-training, it improves the agent's exploration strategy—how to search, how many branches to open, when to cut losses, and how to allocate compute.
The core mechanism turns each task's full exploration history into a Replay Simulator where the agent "dreams", testing counterfactual hypotheses (different branch order, more parallel tasks, earlier abandonment). Since historical results are already computed, these experiments are nearly free, forming a loop of explore → accumulate history → dream → improve strategy → explore again. On four GPU Kernel tasks, the best case reaches equal performance with only 41% of baseline iterations (1/2.43), and up to 2.09x performance at matched budget.
A notable ablation: summarizing past experience back into the prompt made things worse—the distilled "lessons" became biases that pushed the agent toward a few plausible directions too early, hurting exploration diversity.
More from coding & agent
- Dev generates a full software promo video in pure code with GPT-6 Astra, no generative models — op7418 · 2026-09-17
- Frida, an iOS Personal Life Agent, Opens Second TestFlight Batch — Scobleizer · 2026-09-17
- Doubao Seed-2.1-pro-0915 hands-on: tool-calling score jumps 38.4% to 64.8%, coding cost down 40% — vista8 · 2026-09-17
- Dev claims harness makes agent computer use consume identical tokens to regular tool use — TejasKumar_ · 2026-09-17
- Spreadsheet agents should deliver editable drafts, not just approval summaries — Mountain_Athlete1350 · 2026-09-17
- Score Studio Launches as All-in-One Vision AI Platform for Annotation, Training and Deployment — markjeffrey · 2026-09-17