Masked Visual Actions turns 15 hours of robot video into a zero-shot world model
jbhuang0604 · x · 2026-07-24
Masked Visual Actions turns video models into robot world models
Researchers introduce Masked Visual Actions (MVA), a way to repurpose pre-trained video models as robot world models. The core idea is to express actions in the language of vision, letting the model answer counterfactual questions such as: if an object moves in a certain way in the clip, how will the scene evolve?
The method connects two robot-relevant capabilities:
- Forward dynamics: predict future observations from observations and actions
- Inverse dynamics: infer actions from observation sequences
According to the authors, the approach generalizes zero-shot to unseen embodiments and scenes despite being fine-tuned on only 15 hours of single-robot data. They also say the same representation works whether the “thing” is a robot or an object, which makes it useful for model-based planning, policy evaluation, and action inference.
Related event: Zero-Shot Robot World Model Trained with 15 Hours of Video(3 posts)→
More from Embodied
- ROS names Alejandro Hernández Cordero new project lead, eyes 2027 release — ___Mufasaa · 2026-10-27
- ROS Meetup coming to Google's NYC office Oct 29 with speakers from Tangent, Cohesive Robotics, Intrinsic — kscottz · 2026-10-27
- Tesla officially expands robotaxi service area across much of Houston, Texas — Polymarket · 2026-10-27
- Walden Robotics teaches robots to call for help, blending remote assist with autonomy — adnothing · 2026-10-27
- Folding-screen hinges: dozens of ultra-precise parts, one grain of sand can kill the panel — BenBajarin · 2026-10-27
- No π0 weights changed: Motoniq layer lifts real-robot precision fitting from 0/10 to 9/10 — hbouammar · 2026-10-27