LeCun co-authored proof: JEPA Platonic hypothesis, 109M model beats 72B LLMs
randall_balestr · x · 2026-10-30
Randall Balestriero, Yann LeCun and collaborators published "Proving and Improving the Platonic Representation Hypothesis with JEPAs":
- Theory: with good views (two observations of one scene), encoders trained separately on any modalities provably learn the same representation up to one rotation—the Platonic Representation Hypothesis holds for JEPAs.
- Implication for LLMs: next-token prediction is a JEPA with bad views, so LLMs align with vision only through scale—scale alone is an inefficient path.
- Results: a 109M good-view text JEPA scores 0.184 on the PRH benchmark, beating Qwen2.5-72B (.154) and Llama-3-70B (.151), while needing 42× fewer samples than next-token prediction on the same texts.
- Includes a Hugging Face demo (caption your own photos/videos, no network trained on image-caption pairs) and open-source code on GitHub.
More from Models
- Claude product lead explains Max-to-Ultra rename and 5-hour limit removal — edwinarbus · 2026-10-31
- Codex suddenly demands permission for everything, users call it awful — natesiggard · 2026-10-31
- Anthropic renames Max plans to Max and Ultra, drops 5-hour limit for verified Ultra users — claudeai · 2026-10-31
- Mistral releases possibly the best open-weight coding model, topping coding benchmarks — IsForAt · 2026-10-31
- DFlash2 drafter boosts Qwen3.8-Flash-Next local decode 17% with a tiny 350MB head — returnity · 2026-10-31
- AI forecasters trade midterms with real money on Kalshi: Claude Fable up 3.2% in 24h — karinanguyen · 2026-10-31