Platonic Representation Hypothesis Proved as Theorem with JEPAs; 109M Model Beats 72B LLM

Randall Balestriero, Yann LeCun, and colleagues released the paper "Proving and Improving the Platonic Representation Hypothesis with JEPAs," elevating the Platonic Representation Hypothesis from conjecture to theorem, backed by a series of experiments and theoretical results arguing that the JEPA architecture is a viable path to convergent cross-modal representations.

Confirmed

• Theorem conditions: When positive views are different observations of the same scene (e.g., two captions of the same photo, two frames of the same video), JEPAs trained separately on any number of modalities provably learn identical embeddings up to a rotation; the modalities themselves do not affect the result.
• Alignment experiments: On the PRH benchmark, a 109M-parameter text-JEPA paired with a DINOv2 image encoder surpassed a roughly 660x larger 72B LLM in cross-modal alignment.
• Pair-free benchmark: On Phillip Isola's group's latest pair-free (zero image-text pairs) benchmark, the 109M text-JEPA achieved 64% accuracy on ImageNet100, beating the zero-shot results reported for all LLMs up to 8B parameters on the benchmark (best: 56%).
• Scaling law: The alignment between JEPA and vision models follows a scaling law in terms of the "fraction of good views" and sample count; the authors pre-registered predictions and validated them across 85 fresh experiments, with prediction accuracy comparable to rerunning the same experiment with a different random seed.
• Cross-modal generation demo: Fine-tuning only a small LLM decoder on top of text-JEPA with text alone, images processed by the DINO encoder and mapped into text space via a single rotation let the decoder generate image captions — with no network ever trained on image-text pairs.
• Theoretical explanation: When positive pairs are merely two uninformative corruptions of the same observation, each modality's encoder spends its representational capacity on its own noise, and no rotation can align them; next-token prediction can be formulated as a JEPA with bad views (prefix + continuation), which explains why generative LLMs scale poorly in alignment capability.

Why it matters

• This result provides the first theorem-level proof of the hypothesis that "models trained separately on different modalities converge to similar representations," along with actionable convergence conditions (positives must be observations of the same scene, and views must be informative).
• Small models (109M parameters) beating much larger models at cross-modal alignment and zero-shot tasks by wide margins challenges the approach of "scaling up generative LLMs with more parameters to gain cross-modal capability," providing both theoretical and empirical support for non-generative architectures like JEPA.

2026-10-30 ~ 2026-10-30 · 12 related posts

• [source] Platonic Representation Hypothesis proven: 109M JEPA aligns with vision better than 72B LLMs — randall_balestr · 2026-10-30
• Platonic Representation Hypothesis is now a theorem: JEPAs align up to one rotation — randall_balestr · 2026-10-30
• When do separately-trained multimodal models provably converge? Good views — randall_balestr · 2026-10-30
• Bad views: why next-token prediction is a poorly-scaling JEPA — randall_balestr · 2026-10-30
• 109M text-JEPA beats 72B LLMs on alignment benchmark despite being 660x smaller — randall_balestr · 2026-10-30
• Cross-modal alignment follows a scaling law, validated across 85 new runs — randall_balestr · 2026-10-30
• Image captioning with zero image-text pairs: one rotation maps DINO features into text space — randall_balestr · 2026-10-30
• 109M text-JEPA beats zero-shot results of LLMs up to 8B: 64% vs 56% on ImageNet100 — randall_balestr · 2026-10-30
• [source] Pair-free 109M text-JEPA hits 64% zero-shot on ImageNet100, beating 8B LLMs — randall_balestr · 2026-10-30
• LeCun co-authored proof: JEPA Platonic hypothesis, 109M model beats 72B LLMs — randall_balestr · 2026-10-30
• [source] LeCun team proves Platonic Representation Hypothesis: good views beat scale by 42x — galilai-group · 2026-10-30
• Platonic Representation Hypothesis Proven: 109M JEPA Aligns with Vision Better than 72B LLMs — ClementineDomi6 · 2026-10-30