Google's Dream-RSI lets agents 'dream' over replayed exploration history for self-improvement
agihouse_org · x · 2026-09-16
Google and Google DeepMind released Dream-RSI (Recursive Self-Improvement through Evolving Worlds), an RSI route that doesn't touch model weights — instead the agent improves its own exploration policy.
Key mechanism:
- Every task run logs a full exploration history: branches tried, failures, compute spent.
- Dream-RSI turns this history into a Replay Simulator where the agent "dreams" — running counterfactual experiments (different branch first, more parallelism, earlier cutoffs) at near-zero execution cost, since outcomes are already computed.
- Better policies get redeployed, closing a loop: explore → accumulate history → dream → improve policy → explore again.
Results: on GPU kernel tasks, VGG16 and LayerNorm hit similar performance with 2.43× and 1.79× fewer generations; ConvDiv and ConvMax improved performance 2.09× and 1.44× at similar budgets.
Counterintuitive finding: summarizing past experience into prompts for the next round performed worse — those summaries become biases that collapse exploration diversity. The paper's takeaway: before learning to modify its weights, an agent can first learn to modify its harness.
More from AGI Musings
- vikhyat predicts alternating compute oversupply and undersupply on the road to AGI — TheZachMueller · 2026-11-02
- Sam Altman: AI has surpassed top scientists, and 2026 will be a turning point year — kimmonismus · 2026-11-02
- DeepSeek design doc: a person can only follow 3-5 agent sessions at once, attention is the real bottleneck — heyshrutimishra · 2026-11-02
- AI researcher: we have no reliable way to tell whether an AI system is conscious — dioscuri · 2026-11-02
- Toby Ord: track both what models can and can't do to see real capability gaps — tobyordoxford · 2026-11-02
- Forward Deployed Verifier will soon be as valuable as FDE, argues poster — vaibhavbetter · 2026-11-02