La-JEPA: Compact Robot Policies with Latent World Modeling
Mateo Clémente, George Z. Xue, Amin Abyaneh, Liheng Ma, Rui Heng Yang, Tongtong Cao, Zhanguang Zhang, Yingxue Zhang
cs.RO, cs.CV, cs.LG
2026-11-05
A compact policy transformer trained from scratch in frozen V-JEPA 2.1 features beats 12.4B generator-backed policies on LIBERO-Plus and RoboTwin 2.0, at up to 5x lower latency.
Most robot manipulation policies inherit a generative backbone. World-action models (WAMs) fine-tune pretrained video generators; vision-language-action (VLA) models attach action heads to pretrained VLMs. Either way, the policy's architecture, tokenization, and inference cost are fixed by a transformer designed for synthesis, with deployments running from a few billion to over 12B parameters. Video VAE latents also spend much of their capacity on pixel fidelity that control never asked for. The paper asks a blunt question: drop the generative backbone, train a compact transformer from scratch, and see where it lands.
La-JEPA initializes its policy transformer randomly; a frozen V-JEPA 2.1 ViT-L encoder supplies every visual feature. There is no RGB pathway and no pixel reconstruction anywhere in the model.
Sizes S/B/L carry 0.10/0.34/1.14B trainable parameters (0.74/0.98/1.78B deployed with frozen encoders), against 2.0 to 7.8B trainable for the baselines on the same target demonstrations.
| Method | Deployed params | LIBERO-Plus (corrected) | RoboTwin clean/randomized | Latency (10 steps) |
| La-JEPA B | 0.98B | 87.3% | 86.2 / 35.4 | 25.2ms |
| La-JEPA L | 1.78B | 88.0% | 86.0 / 32.1 | 59.7ms |
| NowWAM | 7.1B | 86.2% | — | — |
| Fast-WAM | 12.4B | 70.8% | 77.8 / 1.9 | 88.1ms |
Among policies without embodied pretraining, all three sizes lead RoboTwin 2.0 in both regimes, with the widest margin in randomized scenes, where Fast-WAM falls to 1.9%. On LIBERO-Plus with corrected instructions, L leads NowWAM by 1.8 points at a quarter of the deployed parameters. On four physical Franka tasks, the from-scratch B policy averages 17.50/20 against the embodied-pretrained π0.5 at 17.25, posts the best Push T score of any compared policy, and the S size runs roughly 5× faster than Fast-WAM.
Two ablation numbers deserve memory. Swapping V-JEPA features for the Wan2.2 video VAE latents that underlie the Fast-WAM family costs about 16 points on both RoboTwin regimes, and the VAE policy collapses to 1.0% in randomized scenes. Dropping training-time future prediction costs 7.0 points on RoboTwin and 2.1 on LIBERO-Plus across three seeds (McNemar p<0.01). Future prediction runs only at training, but the policy is measurably worse without it.
The paper is direct evidence against the assumption that robot foundation models must inherit a generative architecture: a control-shaped 1B-class model wins on both quality and speed. The practical pull is cost. 0.34B trainable parameters and 25ms policy calls close the loop on a single consumer GPU, with training confined to the target demonstrations. There is also a benchmark-hygiene finding: the released LIBERO-Plus evaluator appends perturbation parameters to the instruction text in 8,493 of 10,030 variants, hitting video-generator WAMs hardest (Fast-WAM gains 19.3 points once corrected), so prior rankings on this benchmark deserve a second look.
The authors' own list: no systematic hyperparameter search, so the non-monotonic RoboTwin scaling (59.7/60.8/59.0 across S/B/L) goes unexplained; training uses only target demonstrations, so behavior under embodied pretraining is untested; it still trails the strongest embodied-pretrained policies on randomized RoboTwin scenes; the real-robot evidence is four tasks on one Franka setup.
Points worth holding in reserve: most design ablations use a single training run (only the LIBERO-Plus comparison runs three seeds); the feature-space study compares representations bundled with their interfaces into the policy, not representations in isolation, and one clean-scene comparison favors Wan2.2; JEPA-WAM's RoboTwin numbers cover only 20 tasks, with the 50-task figures extrapolated by linear regression; and every baseline re-evaluation runs in the authors' own harness, under a protocol the baselines did not get to tune against.