What Makes World Action Models Generalize? An Empirical Study of Test-Time Future Modeling
Renping Zhou, Zanlin Ni, Zihao Fan, Guohao Fu, Zeyu Liu, Hao Shi, Jie Zhang, Chi Bene Chen, Yang Yue, Xueyang Fu, Gao Huang
cs.CV, cs.RO
2026-09-28
Latent WAMs match in-distribution but drop off-distribution. Simple-WAM uses one fully noised future pass, scoring 79.5 vs 67.7 on LIBERO-Plus at 74.7 ms (3.8× vs explicit).
World Action Models (WAMs) initialize from a video generator and learn to predict both future frames and motor commands. Explicit WAMs denoise those future frames to clean pixels at every action chunk so the action expert can read them. Video tokens dwarf action tokens, so that denoising dominates latency and gets in the way of high-rate closed-loop control.
Latent WAMs keep the video loss at training time and drop the future tokens at test time, running the video expert once on the current frame only. Under a matched Wan2.2-5B backbone, data, and budget, in-distribution success is essentially tied: 97.75 vs 96.85. That number is what the efficiency papers cite when they treat future generation as a training objective, not a test-time need.
WAMs were motivated by generalization: lighting and layout shifts, few demonstrations, and skills read off action-free video of a new task. Those three axes go into one protocol here. In-distribution parity does not transfer. Once the action expert stops reading future representations, all three axes fall.
The backbone follows Fast-WAM: Wan2.2-5B video DiT plus a 1B action expert interpolated from it. The only structural difference between explicit and latent is an attention mask that lets or blocks action tokens from attending to future video tokens. Chunk size H=32, 9 video frames, K=10 action denoising steps.
Test-time future conditioning is required for generalization. On LIBERO-Plus, explicit 67.72 vs latent 53.75. At 10 demos per task, 96.95 vs 88.50. After feeding action-free video of held-out tasks into the video branch only, explicit climbs from 6.25 to 69.90; latent goes from 2.10 to 5.90. A skill learned from action-free video almost does not hold for the latent setup.
A clean future is unnecessary. Take an explicit checkpoint and run the video expert for only the first tdenoise steps, then freeze that feature for the rest of the chunk. At tdenoise=1 and τ=1, the future slots are pure Gaussian noise. That single pass already beats latent by 26.44, 8.2, 13.4, and 89.8 points on the four settings. The remaining nine denoising steps move the score by at most -1.62 to +1.8. The gain is in preparing the future slots, not in rendering the future.
Simple-WAM keeps those two facts and nothing else. At inference, future tokens stay at noise, the video expert runs once, and that feature conditions all K action steps. Cost is Cvid+K Cact, sitting between explicit K Cvid+K Cact and latent Ccur+K Cact. At training, flow time is τ=1 with probability p=0.5 and otherwise the original schedule, because inference only ever reads τ=1 while a standard explicit budget barely spends time there. Endpoints fail: p=0 restores the train/test mismatch, p=1 leaves the pretrained video objective; three-axis averages drop to 90.8 and 88.6. From p=0.25 to 0.75 the average moves by 1.3 points.
Replacing future tokens with learned queries or zeros, while keeping the video diffusion loss, drops the three-axis average from 94.2 to 52.7 and 54.9. Noised tokens are what the video backbone was pretrained to denoise. Queries and zeros are not.
Among models without embodied pretraining, weighted LIBERO-Plus success looks like this.
| Method | LIBERO-Plus | Task gen no/with video | Latency |
| FastWAM (latent) | 53.8 | 2.1 / 5.9 | 62.0 ms |
| FastWAM-Joint (explicit) | 67.7 | 6.3 / 69.9 | 286.9 ms |
| Simple-WAM | 79.5 | 10.1 / 73.6 | 74.7 ms |
π0.5, which does have large-scale embodied pretraining, sits at 84.4 on the perturbation axis, 4.9 above Simple-WAM. Mean success across the four generalization settings in Fig. 4: Simple-WAM 65.1, explicit 60.2, latent 37.6, π0.5 48.7. Few-shot: Simple-WAM hits 92.4 / 97.2 on LIBERO 5-shot / 10-shot, comparable to explicit, and 37.2 on RoboTwin 10-shot where latent falls to 4.8. RoboTwin task transfer is 6.2 vs explicit 5.4 without video and 44.5 vs 47.3 with video, the one Tab. 2 cell explicit still wins.
Latency on one RTX 5090, each method under its own schedule: Simple-WAM 74.7 ms per chunk, explicit 286.9 ms (3.8× slower), latent 62.0 ms (12.7 ms gap).
Real robot: AgileX Aloha, four training tasks, 200 demos each, 30 trials. In distribution the three models sit within 2.3 points. Under lighting, layout, and background shifts, latent drops to 25.2 while Simple-WAM holds 69.2. At 50 demos per task: 69.6 vs latent 37.2. Held-out Store in Order plus action-free video: Simple-WAM 2.2 to 87.8, explicit 68.9, latent 10.0.
If you already train a WAM, stop treating full video denoising as a deployment requirement. Those extra nine steps buy visual fidelity the action expert does not use. Dropping the future at test time, the latent recipe, also drops the generalization that was the point of WAMs, and in-distribution scores will not warn you.
The patch is thin: an attention mask and a mixed τ. It slots into a Fast-WAM-style stack. You pay one video forward with future slots, +12.7 ms over latent, 3.8× faster than explicit. The same split shows up on a real dual-arm.
This is a mechanism result plus a small inference/training alignment fix, not a new backbone or a new dataset. π0.5 with embodied pretraining still leads the perturbation axis by 4.9. With zero data of a new task, all three stay near the floor.
The paper is explicit: everything here is a 5B backbone with no large-scale embodied pretraining. Whether the same first-step-is-enough curve holds at larger scale, or after embodiment pretraining, is open.
Task generalization without any data of the held-out task is 10.1, not a skill that appears from nowhere; the useful number is the action-free-video setting. On RoboTwin with video, explicit still leads 47.3 to 44.5. Real-world evaluation is four tasks and 30 trials each. Camera perturbation is 55.6 for Simple-WAM vs 78.4 for π0.5. The controlled comparison is tied to Fast-WAM's structured mask; generate-then-act stacks and other video backbones were not swept.