Alibaba DAMO releases RynnWorld-Latent: cross-embodiment world model pretrained on 553.5M frames
Alibaba-DAMO-Academy · hf · 2026-10-28
Alibaba DAMO Academy, with Hong Kong Embodied AI Lab, CUHK, and Hupan Lab, released RynnWorld-Latent, a cross-embodiment video world model tackling a key problem: robot action spaces are incompatible and human videos lack native control labels.
How it works:
- Uses 3D-grounded latent actions (introduced by RynnLAM) as a shared conditioning interface, enabling joint pretraining on human interaction videos, robot demonstrations, and simulation rollouts without a common native action space
- Built on a pretrained video transformer, combining global action tokens with an identity-initialized local pathway that aligns actions to future video latents
- Pretrained on 23 datasets totaling 553.5M frames, then adapted to target robots via post-training
Two data-generation routes: digital teleoperation (target-robot commands generate action-video demos) and latent-action synthesis (source motion transferred onto an edited initial frame with retargeted pseudo-robot actions).
Results: improved video prediction metrics on public benchmarks and two real platforms; synthetic mid-training raises policy success rates on LIBERO and LIBERO-Plus, and both routes improve real-world policy success.
More from Embodied
- Dyna Robotics unveils Taku robot with a craftsman-first video, no spec sheet — heyshrutimishra · 2026-10-28
- PDRVLA adds physical dynamics tokens to VLA models, topping LIBERO with 99.3% — Hanqing Wang · 2026-10-28
- Developer hooks GPT into a Hasselblad: focus, shutter and chat, open-sourced — rrRadium · 2026-10-28
- Robots Coming for Your Jobs: A Playful Reaction to a Demo — alysha_lobo · 2026-10-28
- New benchmark tests whether home robots navigate more cautiously around babies than trash bins — SeoulNatlUniv · 2026-10-28
- Early adopter revisits AI hardware: still meh, runs hot without opening apps — alexmacgregor__ · 2026-10-28