Google's Dream-RSI lets agents 'dream' over replayed exploration history for self-improvement
agihouse_org · x · 2026-09-16
Google and Google DeepMind released Dream-RSI (Recursive Self-Improvement through Evolving Worlds), an RSI route that doesn't touch model weights — instead the agent improves its own exploration policy.
Key mechanism:
- Every task run logs a full exploration history: branches tried, failures, compute spent.
- Dream-RSI turns this history into a Replay Simulator where the agent "dreams" — running counterfactual experiments (different branch first, more parallelism, earlier cutoffs) at near-zero execution cost, since outcomes are already computed.
- Better policies get redeployed, closing a loop: explore → accumulate history → dream → improve policy → explore again.
Results: on GPU kernel tasks, VGG16 and LayerNorm hit similar performance with 2.43× and 1.79× fewer generations; ConvDiv and ConvMax improved performance 2.09× and 1.44× at similar budgets.
Counterintuitive finding: summarizing past experience into prompts for the next round performed worse — those summaries become biases that collapse exploration diversity. The paper's takeaway: before learning to modify its weights, an agent can first learn to modify its harness.
More from AGI Musings
- A third theory on CEOs' AI doom warnings: regulation as a scapegoat for plateauing progress — Pretend-Pie9487 · 2026-09-16
- Timothy Lee: how should law treat negligent releases of malicious-seeming AI? — binarybits · 2026-09-16
- No AGI means 100% asteroid doom: the quip puncturing p(doom) debates — seanmcdonaldxyz · 2026-09-16
- Author hits back at Alpha School supporters after AI boot camp exposé — benjaminjriley · 2026-09-16
- Washington Post podcast: Tim Lee on what AI experts fear most — binarybits · 2026-09-16
- Reddit 热议:对齐未解前暂停前沿模型,安全措施反促更强模型 — igpila · 2026-09-16