LeCun explains JEPA theory: understanding over knowledge accumulation means far fewer parameters
ylecun · x · 2026-11-02
Responding to a paper where 109M parameters beat 72B, Yann LeCun lays out the theory: LLMs stack declarative knowledge, which is why they need so many parameters and memory. JEPA world models instead replace knowledge accumulation with understanding — the underlying similarity structure of the data (word co-occurrence in text, object co-occurrence in images) uniquely determines the shape of the representation point cloud when the encoder is trained with anti-collapse criteria like SIGReg or VCReg rather than reconstruction. That's why such models can be useful with far less memory and training data. Context: LeCun left Meta in November and started his own lab in Paris.
More from Models
- How LLM inference actually works: from 140GB weights to GPUs and model serving, explained as a story — _jaydeepkarale · 2026-11-04
- $100 of GPU time edits model weights to cut thinking tokens by 40% — jarrodwatts · 2026-11-04
- Sarvam open-sources Prism, a 44ms decision model that skips text entirely — heyshrutimishra · 2026-11-04
- Frontier models' teaching ability is 'off the charts', says dev: your 'I know kung fu' moment — shauseth · 2026-11-04
- kalomaze: crude diffusion autoencoders may lose low-frequency info at the latent stage — kalomaze · 2026-11-04
- Dev complains frontier models can't do light-touch English editing — A_K_Nain · 2026-11-04