Policies trained fully inside world models deploy zero-shot on real robots, beating PPO
chris_j_paxton · x · 2026-09-04
A new preprint by Joseph Amigo, Rooholla Khorrambakht, Nicolas Mansard, and Ludovic Righetti trains RL policies entirely inside learned world models — no simulator — and deploys them zero-shot on real robots.
The problem: model-based RL in world models is usually intractable, since accurate gradients require capturing contact dynamics with a very large model.
Method: a decoupled first-order gradient (FoG) RL scheme —
- a large global world model (DIAMOND for Push-T, DreamerV4 for ego-centric tasks) generates high-fidelity forward trajectories;
- a lightweight latent-space surrogate approximates its local dynamics for efficient gradient computation;
- both the world model and policies are learned from scratch from real-world data.
Results: zero-shot real-world deployment on a tabletop manipulator (Push-T), a Unitree G1 humanoid (ego-centric grasp and lift), and a Go2 quadruped (push cube). On the canonical real-world Push-T benchmark, sample efficiency is significantly better than PPO, with similar gains on ego-centric manipulation.
Related event: Robots learn contact-rich manipulation in world models without simulators(2 posts)→
More from Embodied
- Sentdex trains robots to go from lying down to a half kneel, eyeing a full stand — Sentdex · 2026-10-21
- NVIDIA open-sources MotionBricks: 350K motion skills at 15,000 FPS, 2ms latency — aftahi_ai · 2026-10-21
- NVIDIA Cosmos3-H-Dreams: Generative Simulation Brings Clinical Surgery to Life — Hugging Face Blog · 2026-10-21
- FLoTS-SLAM uses optical-flow residuals to hit 0.73 DM-IoU in dynamic monocular SLAM — zhenjun_zhao · 2026-10-21
- Plato ring launches at $199 to turn near-silent speech into AI commands — TinfoilTricorn · 2026-10-21
- Dream-1 controlled 31 unseen robot types across 53 countries within 24 hours — gharik · 2026-10-21