LeCun paper: text JEPA aligns with DINOv2 views using 42x fewer samples
jiqizhixin · x · 2026-11-19
A new arXiv preprint by Randall Balestriero and Yann LeCun (Brown, NYU, AMI Labs) asks whether a text encoder trained separately can learn representations aligned with a vision encoder. Bottleneck: captions mix scene information with stylistic choices, so masking one caption leaves both in the signal. Mechanism: train a text JEPA on distinct captions of the same image; the paper proves alignment up to rotation in an idealized setting. On WIT-1024, a 109M text JEPA reaches 0.184 mutual 10-NN alignment with DINOv2 ViT-B/14 versus 0.154 for Qwen2.5-72B — far smaller models achieve better alignment with 42x sample efficiency.
More from Research
- ZK Podcast: ARIA's Nicola Greco on generative cryptography and AI scaling trust — DavideCrapis · 2026-11-19
- Why ViTs beat CNNs: 113k-citation paper shows scale trumps inductive bias — moschles · 2026-11-19
- StatePortal synthesizes agent checkpoints for GUI deep-state testing, 0%→27.5% completion — SJUT1 · 2026-11-19
- SADL plug-in loss claims to cut JEPA training collapse from 50% to 0% — TheTuringPost · 2026-11-19
- PNAS Study: LLMs Amplify Gender Bias Both Writing CVs and Screening Them — AnnaCiaunica · 2026-11-19
- Idea: native-precision quant for prefill, higher-accuracy weight-only quant for decode — AlpinDale · 2026-11-19