AgentGarten: Code Worlds with a Real-Time Neural Renderer for Evolving Agents
Scobleizer · x · 2026-10-09
A new framework, AgentGarten (arXiv:2610.12374, open-source), builds real-time interactive virtual worlds for agents to learn through exploration.
- It couples simulators/game engines with a shared neural renderer: simulation backends keep persistent world state and run program-defined rules, while the renderer generates visual observations from structured conditions.
- The renderer adapts a pretrained video model to geometry conditions and distills it via Adversarial Forcing, which makes history prefilling differentiable through exact replay and adds adversarial supervision; inference is optimized for real-time interaction.
- Agents improve by distilling each round of experience into playbooks that later agents inherit and refine.
- In a hide-and-seek demo, agents build shelters by round 4 and ramps by round 10.
More from Embodied
- AI Smart Cap With Neural Sensing, Camera and Voice AI Previewed for CES 2027 — ThePeterMick · 2026-10-09
- Robot Girl Gang: A New Blog of Sharp Takes from Top Female Roboticists — mattbeane · 2026-10-09
- Watch: Robots folding hotel laundry end to end — LexiLove · 2026-10-09
- Installer uses Apple Vision Pro on the job, seemingly seeing pathways through walls — tristanbob · 2026-10-09
- Only Tesla and XPeng Join Paris Motor Show Real-World Intelligent Driving Demo — Scobleizer · 2026-10-09
- Robo dir Aggregates 3,494 Robotics Datasets With 236B Frames and an MCP Server — Scobleizer · 2026-10-09