LeVJEPA from LeCun's team matches V-JEPA 2 with up to 20.8x less training compute

TheTuringPost · x · 2026-08-30

Yann LeCun, Randall Balestriero and colleagues released LeVJEPA, extending LeJEPA's collapse-free objective to video pretraining, with paper, code and project page public.

Method

Minimal architecture: a single encoder plus a small projector (discarded after training) — no EMA target encoder, stop-gradient, or capacity-limited predictor; the objective has a single hyperparameter.
It builds one global and several cropped/augmented views of the same 16-frame clip, applies an invariance loss across views, and uses SIGReg regularization with a provable collapse-free guarantee.

Efficiency tricks

Only 5% of video tokens are processed during training; the rest are randomly dropped, cutting compute while improving downstream accuracy.
Causal temporal attention: each frame attends only to current and previous frames, suited to streaming video and world models that update understanding incrementally.

Results: At matched epochs on identical data, LeVJEPA matches or surpasses V-JEPA 2 across ViT-S/B/L with 5.6–20.8x less pretraining compute; at matched FLOPs it beats the strongest video baseline by 7.6 points on ImageNet-1K while staying competitive on motion-centric benchmarks.

Original post →

LeVJEPA from LeCun's team matches V-JEPA 2 with up to 20.8x less training compute

More from Models