Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation
Jie Zhang, Xiaoyue Chen, Anzhe Chen, Dayiheng Liu, Deqing Li, Gengze Zhou, Hale Yin, Haoqi Yuan, Haoyang Li, Jiahao Li, Jiazhao Zhang, Jingren Zhou, Kaiyuan Gao, Kun Yan, Lihan Jiang, Ningyuan Tang, Pei Lin, Qihang Peng, Shengming Yin, Tianhe Wu, Tianyi Yan, Xiao Xu, Yan Shu, Yanran Zhang, Ye Wang, Yi Wang, Yilei Chen, Yixian Xu, Yiyang Huang, Yuxiang Chen, Zekai Zhang, Zhendong Wang, Zixing Lei, Zhixuan Liang, Zihao Liu, Zikai Zhou, Chenxu Lv, Xiong-Hui Chen, Chenfei Wu
cs.CV
2026-06-16
An embodied world model that takes a natural-language action and predicts the next physically grounded video. Ranks 1st on EWMBench and DreamGen; perfect on four WorldModelBench physics sub-metrics.
Embodied intelligence needs a proper world model. Not the vague buzzword, but the strict version: given the current frame and an action, predict what the world looks like next. Self-driving has to anticipate traffic, a robot arm has to predict how an object moves after a grasp, navigation has to imagine what is around the corner. It is the same problem dressed up differently.
Existing video generators split into two camps. General models like Veo and Wan produce gorgeous footage but take no action input and ignore physics, so their videos routinely violate basic laws. Robot-specific world models are too narrow, tied to one robot and one action format, with no cross-embodiment generalization. Qwen-RobotWorld tries to do it in one shot: treat natural language as a universal action interface, and have a single model predict physically grounded futures across manipulation, driving, navigation, and human-to-robot transfer.
The core is a 60-layer double-stream diffusion transformer (Double-Stream MMDiT). Two streams run side by side: a frozen Qwen2.5-VL (7B) compresses language and semantics into tokens, while a Wan-VAE (127M) encodes the video latents. Layer-wise joint attention lets the semantic stream condition the video stream. The crucial point is that the language here is not a caption but the action itself. The MLLM encodes an action description into tokens, and those tokens drive the generation.
For data they built Embodied World Knowledge (EWK): 8.6M video-text pairs, 200M-plus frames, spanning 20-plus embodiments and 500-plus action categories, with a 70/30 split between embodied and general data. The core construct is the action-language mapping, pairing a word-level action description with the visual change it produces.
Training runs in two stages (General+Expert progressive curriculum): first learn general visual priors, how the world looks and how objects move, then inject embodied specialization under action conditioning. Both stages share the same language interface.
The numbers are what matter.
| Benchmark | Result | Comparison |
| EWMBench overall | 4.60, 1st | LVP 4.05 |
| EWMBench motion fidelity (HSD) | 0.566 | LVP 0.425, +33% |
| DreamGen Bench | 4.952, 1st | none |
| WorldModelBench overall | 8.99, 1st open-source, 3rd overall | Wan2.6 9.27, Veo3 9.25 |
| WorldModelBench physics (Newton/mass/fluid/gravity) | 1.00 on all four | penetration 0.94 |
| PBench | 0.804, 1st open-source | domain understanding 0.857 |
On WorldModelBench the closed-source Wan2.6 and Veo3 still lead overall, but Qwen-RobotWorld is the first open-source model to get close, and it posts perfect scores on four physics sub-metrics. The weak dimension is instruction following at 2.33/3.0, below Veo3's 2.52.
A world model that treats natural language as the action interface is the foundation for three things: generating synthetic data for policy training, serving as a virtual environment to evaluate policies, and providing language-guided planning signals for downstream robots. The perfect physics scores matter most, because they suggest the model has internalized physical rules rather than just learning to generate plausible-looking video. For anyone doing embodied data augmentation or simulation-based evaluation, this is the open-source model worth running today.
This is still a technical report. Closed Veo3 and Wan2.6 score higher overall, and the closed loop that would actually prove the thing useful, using predicted futures to train or improve a control policy, is shown only qualitatively.