Rolling-WAM: World Action Models with Rolling Imagination
Yinghua Zhou, Junjie Ye, Yiqi Zhao, Hao Dong, Celina Shiyu Wang, Ruohai Ge, Tingyi Yang, Basile Van Hoorick, Gaurav Sukhatme, Vitor Guizilini, Yue Wang
cs.RO, cs.AI, cs.CV
2026-09-25
Rolling-WAM spreads joint video-action denoising across replans via a staggered-noise window, with 98.1% on LIBERO and 4.5× faster updates than Joint-WAM.
World Action Models (WAMs) generate robot actions together with future video, so the policy can condition on how the scene should evolve. Closed-loop control then dominates cost. After every short burst of motion the robot must replan from the latest camera frame. A standard Joint-WAM denoises the whole prediction horizon from Gaussian noise on every cycle, and every step still runs over the full video-action sequence. Action updates lag.
Fast-WAM turns off future-video generation at test time. Latency falls, and the explicit visual context falls with it. Receding-horizon control also discards the unexecuted tail of the window. That tail already informed the current action and could have been refined on the next cycle, not redrawn from noise.
Rolling-WAM applies rolling diffusion to joint video-action generation. Rolling diffusion is a sequence sampler that keeps a sliding window whose noise rises toward the future, so nearby tokens finish first. The horizon is split into W aligned chunks; each chunk holds a video latent span plus K actions. Defaults are W=5, K=16, horizon 80.
Each replan fully denoises the imminent chunk and executes it, only partially refines farther chunks, then shifts the window. Leftover predictions keep their noise levels, a new Gaussian chunk is appended, and the new camera frame becomes the condition.
Let τ fall from 1 to 0 inside a cycle. Chunk j sits at noise σ((j-1+τ)/W). After the shift, each retained chunk is already at the start noise of its new slot. That matching is what makes the compute reusable. Each chunk still gets N=10 denoising steps in total, spread across W cycles, so a steady-state replan uses two steps. At episode start there is no buffer; an initialization schedule starts the whole window from noise and hands off to rolling mode once the first chunk is clean.
The network is a Mixture-of-Transformers: a video expert from Wan2.2-TI2V-5B and an action expert of about 1B parameters (30 layers, width 1024) interpolated from the video backbone. The VAE and text encoder stay frozen. Every action chunk attends to visual features across the full window; action-to-action attention stays inside a chunk; video tokens do not attend to actions. Training is flow matching on rolling and initialization noise profiles with probabilities 0.8 and 0.2. No extra embodied pretraining.
Near-term plans are forced clean. Farther plans stay drafts and keep being revised as observations arrive.
The comparison set includes VLAs and WAMs. Rolling-WAM has no extra embodied pretraining; π0, Motus, and LingBot-VA do.
LIBERO, 50 rollouts per task:
| Method | Spatial | Object | Goal | Long | Avg |
| π0 | 96.8 | 98.8 | 95.8 | 85.2 | 94.1 |
| LingBot-VA | 98.5 | 99.6 | 97.2 | 98.5 | 98.5 |
| Joint-WAM | 99.6 | 99.4 | 98.2 | 96.8 | 98.5 |
| Fast-WAM | 98.2 | 100.0 | 97.0 | 95.2 | 97.6 |
| Rolling-WAM | 98.2 | 98.0 | 98.2 | 97.8 | 98.1 |
The average sits 0.4 points behind Joint-WAM and LingBot-VA (both 98.5%). On Long it is 97.8% against Joint-WAM's 96.8%.
RoboTwin 2.0, 50 bimanual tasks, 100 rollouts per setting: 93.5% clean, 93.0% randomized, 93.3% average, ahead of LingBot-VA at 92.2%, Fast-WAM at 91.8%, and Joint-WAM at 90.6%.
Steady-state replan latency on one NVIDIA A100 under RoboTwin at 384×320, including visual encoding and denoising, excluding init and warmup, 16 executed actions per cycle: Rolling-WAM 215 ms, Joint-WAM 978 ms (about 4.5×), Fast-WAM 548 ms (about 2.5×). π0.5 takes 296 ms and GR00T N1.7 takes 285 ms. Rolling-WAM still maintains an 80-step future; the baselines predict 16. Timing uses no torch.compile or TensorRT.
Unitree G1, one policy on three tasks, 50 demonstrations each, 20 trials:
| Method | Doll Placement | Plate Stacking | Bead Pouring | Avg |
| π0.5 | 55.0 | 70.0 | 60.0 | 61.7 |
| GR00T N1.7 | 75.0 | 65.0 | 60.0 | 66.7 |
| Fast-WAM | 85.0 | 80.0 | 60.0 | 75.0 |
| Joint-WAM | 70.0 | 100.0 | 65.0 | 78.3 |
| Rolling-WAM | 85.0 | 100.0 | 70.0 | 85.0 |
Window sweep on six RoboTwin tasks: 78.2% at W=5, 77.3% at W=3, 69.5% at W=8. Cross-chunk action attention drops those six tasks to 76.3% and LIBERO to 97.9%. A random noise mix reaches 78.5% on the six tasks and 97.3% on LIBERO.
This is a sampling-schedule change. The backbone is still a pretrained video DiT plus an action Transformer. Explicit future video stays in the loop, and replan latency falls below Fast-WAM and two VLA baselines. For teams already running joint video-action denoisers and blocked on closed-loop delay, the method stacks with compression, distillation, and async context, which the paper lists as compatible.
On the humanoid the average is 85.0%, 6.7 points above Joint-WAM's 78.3%. The paper reports smoother chunk boundaries and fewer pauses. LIBERO is a near tie, not a win. The default recipe is N=10, W=5, 16 actions per chunk, a 5B-class Wan video expert plus a 1B action expert. Reproduction is not cheap.
The authors say window and chunk sizes were not swept across different dynamics or control rates, and that retained predictions can lag sudden scene changes, especially with long windows. Adaptive windows and async execution are left open.
The real-world gap sits on three tasks and 20 trials each. Rolling-WAM beats Joint-WAM on RoboTwin and trails by 0.4 points on LIBERO, with no analysis of why full joint denoising suffers more under bimanual randomization. Latency is A100, fixed resolution, uncompiled; the 4.5× may shrink on an edge stack or after a compiler. The 78.2% ablation number is six tasks, not the 50-task 93.5%.