Alibaba Uses World Models for Digital Teleoperation
量子位 · wechat · 2026-07-17
Alibaba DAMO Academy proposed RynnWorld-Teleop, shifting from "human teleoperating a real robot" to "human gestures driving a generative world model." The system directly synthesizes first-person robot video in a digital world and automatically generates joint-level action labels.
The article details three key conditions and implementations of this digital teleoperation: first-person robot perspective, traceable actions, and real-time interaction. The corresponding tech includes depth-aware skeleton rendering, progressive cross-domain training, and distilling a bidirectional video diffusion model into a causal student model running at 40FPS in real time.
In experiments, data generated by RynnWorld-Teleop can directly train real robot policies. For bimanual dexterous hand tasks, usable policies were achieved with zero real-world data. When synthetic data was mixed with real-world data, success rates improved further across most tasks, proving the synthetic trajectories closely match real data in both pixels and feature distribution.
More from Embodied
- Tesla expands Robotaxi rides to seven areas, including new Orlando and Tampa zones — elonmusk · 2026-07-22
- Hands-on robotics workshop on Saturday may be the last in-person session before August — StewartalsopIII · 2026-07-22
- NVIDIA pitches World Foundation Models as a way to scale physical AI data generation — MonaJalal_ · 2026-07-22
- RoboMME Podcast Preview: Benchmarking Memory for Robotic Policies — chris_j_paxton · 2026-07-21
- Gritt says an 8-person crew now installs 3,000 to 4,000 solar panels a day — HaktanSuren · 2026-07-21
- A helium-powered flying robot whale aims to be a quiet companion pet — chris_j_paxton · 2026-07-21