μ0 world model learns robot physics from video only, beats π0 by 5 points on RoboCasa365
jbhuang0604 · x · 2026-11-09
Researchers from University of Maryland and Seoul National University will present μ0 at CoRL 2026: a scalable 3D interaction-trace world model (Best Paper at IROS 2026 RoBoWoMo Workshop) that learns a "physical language" for robots purely from video, requiring zero action labels.
- Core idea: predict 3D traces of semantic interaction points (objects, tools, hands, contact regions) instead of pixels or low-level actions, enabling transfer across embodiments from video-only pretraining.
- Why traces: pixel-space video models burn capacity on appearance, while action-labeled VLAs are bound to specific embodiments; μ0 occupies the middle ground.
- Results: μ0 + action expert hits 30.25% average success across 8 RoboCasa365 tasks, outperforming π0 by 5.0 points and TraceGen + action expert by 7.25 points despite video-only pretraining.
Code and dataset are open-sourced.
More from Embodied
- Apple's HALO humanoid trains in 1000 simulated homes, hits 91% success in real houses — heyshrutimishra · 2026-11-10
- Boston Dynamics puts wheels on Atlas to stress-test its whole-body RL controller — CyberRobooo · 2026-11-09
- Guava distills GPT-5.4 into a 4B robot agent with just 2,000 simulated trajectories — jbhuang0604 · 2026-11-09
- Dex-One2Many learns dexterous manipulation from one human video, 95%+ success across 4 hands — jbhuang0604 · 2026-11-09
- Fly-by-Code: coding agents pilot drones via active visual and physical feedback loops — jbhuang0604 · 2026-11-09
- NVIDIA unveils GR00T 2 open-world robot action model, hitting 95% success with 5 demos — CyberRobooo · 2026-11-09