LeCun paper: text JEPA aligns with DINOv2 views using 42x fewer samples

jiqizhixin · x · 2026-11-19

A new arXiv preprint by Randall Balestriero and Yann LeCun (Brown, NYU, AMI Labs) asks whether a text encoder trained separately can learn representations aligned with a vision encoder. Bottleneck: captions mix scene information with stylistic choices, so masking one caption leaves both in the signal. Mechanism: train a text JEPA on distinct captions of the same image; the paper proves alignment up to rotation in an idealized setting. On WIT-1024, a 109M text JEPA reaches 0.184 mutual 10-NN alignment with DINOv2 ViT-B/14 versus 0.154 for Qwen2.5-72B — far smaller models achieve better alignment with 42x sample efficiency.

Original post →

More from Research

Research channel →