Alibaba Uses World Models for Digital Teleoperation
量子位 · wechat · 2026-07-17
Alibaba DAMO Academy proposed RynnWorld-Teleop, shifting from "human teleoperating a real robot" to "human gestures driving a generative world model." The system directly synthesizes first-person robot video in a digital world and automatically generates joint-level action labels.
The article details three key conditions and implementations of this digital teleoperation: first-person robot perspective, traceable actions, and real-time interaction. The corresponding tech includes depth-aware skeleton rendering, progressive cross-domain training, and distilling a bidirectional video diffusion model into a causal student model running at 40FPS in real time.
In experiments, data generated by RynnWorld-Teleop can directly train real robot policies. For bimanual dexterous hand tasks, usable policies were achieved with zero real-world data. When synthetic data was mixed with real-world data, success rates improved further across most tasks, proving the synthetic trajectories closely match real data in both pixels and feature distribution.
More from Embodied
- LIBERO-MAX: Robot Policies Drop 11–26 Points When the World Changes Mid-Task — drmapavone · 2026-10-27
- Photos from ROSCon UK: BBC's robotic whale, da Vinci surgical robot, PR2 and Baxter — kamathsblog · 2026-10-27
- Agility's Digit robot to expand into two more GXO US warehouses from early 2027 — jonstephens85 · 2026-10-27
- Real2Sem2Real turns one phone video into 2,000+ robot demos, matching 150 teleop demos — Scobleizer · 2026-10-27
- Wayve CEO argues end-to-end deep learning should power driving, safety and world models — alexgkendall · 2026-10-27
- Mundo turns a few room photos into an explorable, simulatable 3D world — at 1.5 FPS — mark_k · 2026-10-27