MirroS' AgentGarten pairs code physics with neural rendering to evolve agents at 30fps
量子位 · wechat · 2026-10-08
MirroS released AgentGarten, connecting executable code environments with a real-time neural renderer: code enforces deterministic physics, the renderer turns exported depth/normal "sketches" into photorealistic first-person views at 30+fps (480p).
- In a hide-and-seek rerun, agents writing Python and seeing only rendered frames learned shielding by round 4 and ramp-climbing by round 10—OpenAI's 2019 RL run needed 25M and 100M games respectively.
- Agents write per-round "lab manuals" recording observations, hypotheses and pitfalls, passed to successors as priors.
- The same loop (task → blind trial → manual → archive) shows steady progress across four other worlds (puppy care, narrow-bridge negotiation, cooperative herding, quarry loader).
- Renderer: streaming chunked generation, exact-replay for long-horizon consistency, Adversarial Forcing with real-video discriminators, custom Triton fused kernels + CUDA Graph for real-time throughput.
The team frames this as a step toward Physical RSI—recursive self-improvement grounded in the physical world.
Related event: AgentGarten: Agents Learn in Neural-Rendered Interactive Worlds(2 posts)→
More from Embodied
- KAIST's ME-World tackles multi-agent egocentric world modeling with joint denoising — kaist-ai · 2026-10-09
- Scoble rides Tesla Cybercab: exceeds expectations, plans to buy one to rent out — Scobleizer · 2026-10-09
- Egocentric data collection is all the rage now, says researcher citing an ahead-of-its-time project — gragtah · 2026-10-09
- DreamTrue robot world model cuts interaction defect rate from 48.12% to 6.25%, tops AgiBot challenge — Junyan Li · 2026-10-09
- NVIDIA's USDCraft uses LLM-written programs to build simulation-ready articulated 3D assets — nvidia · 2026-10-09
- SuperNav: ZJU's Agent Harness Lets MLLMs Navigate Any Scene Without Fine-tuning — zju3dv · 2026-10-09