Using Video Models for Robot Control
oier_mees · x · 2026-07-13
At RSS 2026, the author presented their latest work, Mimic-Video: Video-Action Models for Generalizable Robot Control Beyond VLAs.
The core question: What happens if a robot's policy is built on a pretrained video model rather than a static vision-language backbone?
They propose Video-Action Models (VAMs), leveraging rich temporal representations learned from large-scale video data to boost robot learning. The paper claims this approach yields better sample efficiency and training speeds compared to traditional Vision-Language-Action (VLA) models, while also benefiting from ongoing advancements in foundational video models.
The author also mentions they are in Sydney for RSS, presenting first at the Pioneers Workshop and then at the main conference, and welcomes offline chats about robot learning and foundation models.
Related event: Mimic-Video: Video-Action Robot Control Showcased at RSS 2026(2 posts)→
More from Embodied
- π0.5 study: raising demos from 10 to 438 lifts Place Ball in Bowl success from 7% to 83% — stingning · 2026-10-27
- GPT-6 Astra controls dexterous hands zero-shot at just 5.2% success in 500 trials — stingning · 2026-10-27
- HandsOn benchmark: 66 verified tasks and 18,000 human demos for dexterous manipulation — stingning · 2026-10-27
- HandsOn dexterous-manipulation benchmark: best policy 29.1%, GPT-6 Astra just 5.2% — stingning · 2026-10-27
- DexPort: a handheld interface with a real robot hand for dexterous-hand data collection — pabbeel · 2026-10-27
- ROS names Alejandro Hernández Cordero new project lead, eyes 2027 release — ___Mufasaa · 2026-10-27