Native Action-Prior Learning from Videos for World Action Models
Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He
cs.CV, cs.RO
2026-10-02
NAVA-WAM trains its Action-DiT with future-frame flow matching on action-free video, then grounds it with demonstrations: 73.6% on RoboTwin Random and 14 of 15 on a Franka.
A world action model (WAM) learns the next visual state and the robot action in one network. Scaling it stalls on action-labeled trajectories: they are expensive, and a new arm rarely reuses them. Videos with no action labels are much easier to gather, and they already show contact and what the object does afterward.
The two standard uses of those videos both insert an interface. Vision-representation pretraining keeps dynamics inside an encoder, and the policy only meets that knowledge later, on labeled demonstrations. Latent-action methods infer an intermediate variable from neighboring frames and then ground it to real commands, with extra constraints so the variable cannot copy the next frame. Those constraints also shrink what the interface can express. NAVA-WAM removes the interface. A future-frame loss updates the action network's own weights.
The architecture is a Mixture-of-Transformers. A Video-DiT initialized from Wan2.2-5B models visual dynamics. An Action-DiT of about 1 billion parameters, hidden size 1024, emits a continuous action chunk. The experts keep separate weights and share information only in attention.
Pretraining never sees a robot action. The clips come from Open X-Embodiment, AgiBotWorld, and EgoDex: 235,790 episodes, 1,054.3 hours, cut into about 1.41 million segments. A frozen spatiotemporal VAE encodes each segment, and adjacent latents are one transition. The Video-DiT and the VAE stay frozen. Training runs 14,000 steps on 128 GPUs and updates only the Action-DiT.
Each segment holds three token groups: a clean previous state, a noised future state, and action tokens. Segments cannot attend to each other. Action tokens read both visual states, so the transition lives inside the action network and is never written out as a latent-action vector. Noised future tokens also read the action tokens. Future video is trained with flow matching, a velocity field that carries noise back to data along a straight path, and that loss sends gradients into the Action-DiT. The action stream receives pure noise and has no action target. Showing only the noised future blocks the shortcut of copying the clean target out unchanged.
Action labels appear only at post-training. A cold start comes first: target-domain video updates the Video-DiT while the Action-DiT stays frozen, with no new action supervision. Both experts are then trained with flow matching, video loss and action loss weighted equally at 1. Attention becomes asymmetric. Video queries see only the visual stream, action queries see both, and the visual keys and values can be cached. Inference runs one joint forward pass, stores those keys and values, and denoises actions in the Action-DiT alone. Future video is not generated. In simulation the chunk length is 16 and the solver takes 10 Euler steps.
On LIBERO, 50 demonstrations per task give 2,000 trajectories across 40 tasks, with 50 evaluation rollouts each. NAVA-WAM averages 99.0% success, including 99.0% on the long-horizon suite. π0 scores 85.2% on that suite and 94.1% overall. π0.5 averages 96.9%. In-distribution headroom is already small.
LIBERO-Plus rebuilds the tasks into 10,030 test-only instances with shifts in layout, camera, robot initialization, language, lighting, background, and sensor noise. NAVA-WAM's macro average is 83.5%, against 83.1% for Image-WAM, 53.6% for π0, 61.6% for π0-FAST, and 51.5% for Fast-WAM. It leads on robot initialization (65.6%), language (93.8%), and object layout (87.1%). Image-WAM is higher on camera, lighting, background, and noise.
RoboTwin 2.0 is bimanual. Training uses Clean demonstrations only, 50 per task, 2,500 trajectories, and no Random labels. NAVA-WAM scores 88.5% Clean and 73.6% Random: 3.5 points above Image-WAM at 85.0%, the best Clean baseline, and 17.9 points above MV-WAM at 55.7%, the best Random baseline. It is best or tied on 44 of 50 Random tasks and 32 of 50 Clean tasks.
| Method | Setting | Success |
| π0 | LIBERO / LIBERO-Plus | 94.1% / 53.6% |
| π0.5 | LIBERO | 96.9% |
| Fast-WAM | LIBERO-Plus | 51.5% |
| Image-WAM | LIBERO-Plus | 83.1% |
| NAVA-WAM | LIBERO / LIBERO-Plus | 99.0% / 83.5% |
| Image-WAM | RoboTwin Clean / Random | 85.0% / 37.6% |
| MV-WAM | RoboTwin Clean / Random | 84.0% / 55.7% |
| NAVA-WAM | RoboTwin Clean / Random | 88.5% / 73.6% |
| π0.5 | Franka FR3, 15 trials | 53.3% (8/15) |
| DreamZero | Franka FR3, 15 trials | 66.7% (10/15) |
| NAVA-WAM | Franka FR3, 15 trials | 93.3% (14/15) |
On a physical Franka FR3, NAVA-WAM is post-trained on DROID, about 76,000 teleoperated trajectories, and is not finetuned on the test tasks. Three tabletop tasks, five trials each, produce 14 successes. All three policies score 5/5 on putting a banana in a box. Placing a cube to the left of a bowl is 4/5 for NAVA-WAM and 0/5 for both π0.5 and DreamZero. In all 10 baseline failures the cube is grasped, then dropped in the bowl, held above it, or set on the right. Stacking is 5/5 for NAVA-WAM and DreamZero, 3/5 for π0.5. One policy call on an H100 takes 195.7 ms for π0.5, 379.2 ms for NAVA-WAM, and 3,523.9 ms for DreamZero. Compute per action chunk is 4.30 TFLOPs, next to 4.09 for Image-WAM and 3.56 for Fast-WAM, against 67.69 for HALO, which scores 26.4% on Random.
The pretraining ablation uses the same RoboTwin protocol, and 73.6% at 50 demonstrations matches the main Random result. At 10, 25, and 50 demonstrations per task, no pretraining scores 60.5%, 67.6%, 69.7%; representation pretraining scores 58.2%, 60.6%, 68.1%; latent-action pretraining scores 61.6%, 65.0%, 71.2%; NAVA-WAM scores 66.5%, 72.0%, 73.6%. The 72.0% from 25 demonstrations beats every baseline trained with 50, whose best is 71.2%. A ridge probe onto ground-truth actions, using the full probe set, gives R² of 0.329 for the Action-DiT, 0.153 for representation features, 0.203 for CoMo latent actions, and 0.154 for DynaMo. Filling in the action-free corpus raises Random success from 60.5% to 66.5% at 10 demonstrations, and from 69.7% to 73.6% at 50.
The action network is updated by a future-frame loss before it sees a robot action label. Downstream training does not have to translate a vision encoder or a latent action into motor commands. Demonstrations are still required, and so is the target-domain cold start. What changes is how far a fixed label budget generalizes. Against latent-action pretraining the margin is 4.9 points at 10 demonstrations, 7.0 at 25, and 2.4 at 50.
Inference does not roll out future video. A call takes 379.2 ms, about twice π0.5 and about one ninth of DreamZero. From π0 at 94.1% to NAVA-WAM at 99.0%, LIBERO in-distribution is crowded. The 17.9-point gap is on RoboTwin's randomized domain. The gain sits inside joint world-action models: mostly out-of-distribution success, plus a latency that can actually close the loop.
The paper has no dedicated limitations section. The appendix failures are contact failures. In Move Can Pot the arm knocks the can over and keeps executing the transport with an empty gripper until the step limit. In Hanging Mug the left arm sets the mug beside the rack and the right arm never hangs it. The object and the phase are correct. After a bad grasp, the policy does not try again.
The real-robot sample is 3 tasks and 15 trials. Open-loop horizons are also unmatched. NAVA-WAM predicts 32 absolute joint-position steps and executes 24. DreamZero predicts and executes 24 absolute steps. π0.5 predicts 24 relative steps and executes 8. All three run at 15 Hz with a 1,000-step cap. The experiment does not separate how much of 93.3% versus 66.7% and 53.3% comes from the longer open-loop window.
The 1,054.3 hours of pretraining video are still manipulation footage: robot trajectories with actions stripped off, plus egocentric dexterous video. An R² of 0.329 beats the baselines and remains far from an action you could execute. Many main-table baselines are taken from the original papers; the matched rerun is the label-efficiency ablation, and that is the safer reading of the generalization gap. The LIBERO-Plus average is only 0.4 points above Image-WAM, and Image-WAM wins four visual perturbations, so robustness is not uniformly better. The Gaussian argument in the appendix only shows that noise blocks exact copying of the clean future frame. It does not check whether the real network has other shortcuts.