Huawei's La-JEPA trains a 1B robot policy from scratch, beating 12.4B generator-backed baselines

La-JEPA: Compact Robot Policies with Latent World Modeling

Mateo Clémente, George Z. Xue, Amin Abyaneh, Liheng Ma, Rui Heng Yang, Tongtong Cao, Zhanguang Zhang, Yingxue Zhang

cs.RO, cs.CV, cs.LG

2026-11-05

A compact policy transformer trained from scratch in frozen V-JEPA 2.1 features beats 12.4B generator-backed policies on LIBERO-Plus and RoboTwin 2.0, at up to 5x lower latency.

What problem this solves

Most robot manipulation policies inherit a generative backbone. World-action models (WAMs) fine-tune pretrained video generators; vision-language-action (VLA) models attach action heads to pretrained VLMs. Either way, the policy's architecture, tokenization, and inference cost are fixed by a transformer designed for synthesis, with deployments running from a few billion to over 12B parameters. Video VAE latents also spend much of their capacity on pixel fidelity that control never asked for. The paper asks a blunt question: drop the generative backbone, train a compact transformer from scratch, and see where it lands.

Method

La-JEPA initializes its policy transformer randomly; a frozen V-JEPA 2.1 ViT-L encoder supplies every visual feature. There is no RGB pathway and no pixel reconstruction anywhere in the model.

Sizes S/B/L carry 0.10/0.34/1.14B trainable parameters (0.74/0.98/1.78B deployed with frozen encoders), against 2.0 to 7.8B trainable for the baselines on the same target demonstrations.

Results

MethodDeployed paramsLIBERO-Plus (corrected)RoboTwin clean/randomizedLatency (10 steps)
La-JEPA B0.98B87.3%86.2 / 35.425.2ms
La-JEPA L1.78B88.0%86.0 / 32.159.7ms
NowWAM7.1B86.2%——
Fast-WAM12.4B70.8%77.8 / 1.988.1ms

Among policies without embodied pretraining, all three sizes lead RoboTwin 2.0 in both regimes, with the widest margin in randomized scenes, where Fast-WAM falls to 1.9%. On LIBERO-Plus with corrected instructions, L leads NowWAM by 1.8 points at a quarter of the deployed parameters. On four physical Franka tasks, the from-scratch B policy averages 17.50/20 against the embodied-pretrained π0.5 at 17.25, posts the best Push T score of any compared policy, and the S size runs roughly 5× faster than Fast-WAM.

Two ablation numbers deserve memory. Swapping V-JEPA features for the Wan2.2 video VAE latents that underlie the Fast-WAM family costs about 16 points on both RoboTwin regimes, and the VAE policy collapses to 1.0% in randomized scenes. Dropping training-time future prediction costs 7.0 points on RoboTwin and 2.1 on LIBERO-Plus across three seeds (McNemar p<0.01). Future prediction runs only at training, but the policy is measurably worse without it.

Why it matters

The paper is direct evidence against the assumption that robot foundation models must inherit a generative architecture: a control-shaped 1B-class model wins on both quality and speed. The practical pull is cost. 0.34B trainable parameters and 25ms policy calls close the loop on a single consumer GPU, with training confined to the target demonstrations. There is also a benchmark-hygiene finding: the released LIBERO-Plus evaluator appends perturbation parameters to the instruction text in 8,493 of 10,030 variants, hitting video-generator WAMs hardest (Fast-WAM gains 19.3 points once corrected), so prior rankings on this benchmark deserve a second look.

Limitations

The authors' own list: no systematic hyperparameter search, so the non-monotonic RoboTwin scaling (59.7/60.8/59.0 across S/B/L) goes unexplained; training uses only target demonstrations, so behavior under embodied pretraining is untested; it still trails the strongest embodied-pretrained policies on randomized RoboTwin scenes; the real-robot evidence is four tasks on one Franka setup.

Points worth holding in reserve: most design ablations use a single training run (only the LIBERO-Plus comparison runs three seeds); the feature-space study compares representations bundled with their interfaces into the policy, not representations in isolation, and one clean-scene comparison favors Wan2.2; JEPA-WAM's RoboTwin numbers cover only 20 tasks, with the 50-task figures extrapolated by linear regression; and every baseline re-evaluation runs in the authors' own harness, under a protocol the baselines did not get to tune against.

Terms

Source

What people are saying

Related papers

All paper explainers