LeCun coauthors proof of Platonic Representation Hypothesis: JEPA views need 42x fewer samples
ylecun · x · 2026-11-05
Yann LeCun shares a new paper with Brown's Randall Balestiero, "Proving and Improving the Platonic Representation Hypothesis with JEPAs."
Key points:
- Theorem instead of conjecture: Separately trained JEPAs on any modality converge to representations equal up to rotation whenever positive pairs are two good views of the same scene; bad-view misalignment is bounded everywhere.
- Explains LM-vision alignment: Language models align with vision because text is multiple views of one document; alignment scales with size.
- Empirical scaling law: Validated with 145k captions and 940M web images; predicts unseen runs within seed noise.
- Efficiency: Good-view training matches next-token prediction's alignment with 42x fewer samples; a 109M-parameter text JEPA aligns with DINOv2 better than Qwen2.5-72B (.154) or Llama-3-70B (.151), hitting .184 at 200M samples.
- Bonus: A text JEPA captioner reaches 64% zero-shot top-1 on ImageNet-100 with 8B params, beating 56% at 5B.
Takeaway: positive views, not scale alone, are the lever for platonic representations.
More from Research
- Berkeley project uses pretrained base policy's support to guide exploration for fast online adaptation — abhishekunique7 · 2026-11-05
- Researchers explore deep Transformers as neural ODEs with spectral analysis via DMD — PTenigma · 2026-11-05
- Researchers Undo SFT Behavior by Subtracting Weight Updates from Final Checkpoints — burny_tech · 2026-11-05
- Agent benchmark scores hinge on container configs: 6-point swing from resource limits alone — krishnan · 2026-11-05
- USC-led $3.2M NIH project to analyze 94,000 brain scans across 8 mental illnesses — PTenigma · 2026-11-05
- USC's Zeta Law paper uses Riemann zeta function to predict when data is enough for discovery — PTenigma · 2026-11-05