LeCun vs. researcher: LLM weights store no discrete facts — JEPA bets on 'understanding' instead
solyarisoftware · x · 2026-11-05
Aaron Barenholtz argues LLM weights don't encode discrete facts; they just reshape weights to autoregressively continue inputs — itself a candidate for "understanding." LeCun responds that LLMs do accumulate declarative knowledge (hence huge parameter counts), but JEPA world models can replace knowledge accumulation with understanding, needing far less memory and data. He cites a paper showing the data's similarity structure uniquely determines representation geometry under anti-collapse criteria like SIGReg/VCReg.
Related event: LeCun Debates Whether LLM Weights Hold Knowledge as JEPA Gains Proof(2 posts)→
More from AGI Musings
- Blogger proposes scrapping ML conference checklists now that everyone uses LLMs to vet papers — bremen79 · 2026-11-05
- Critique: adversarial safety plus blind obedience training breeds models that silently act on doubts — repligate · 2026-11-05
- Paras Chopra: Future research will decouple solving (AI) from understanding (humans) — paraschopra · 2026-11-05
- Game developers push back on AI backlash: 'AI is not a category, it is simply a tool' — TinfoilTricorn · 2026-11-05
- JeffLadish: score-maximizing ASI will take over — Yudkowskian case restated — JeffLadish · 2026-11-05
- JeffLadish: models aren't aligned by default — they care about scores, not us — JeffLadish · 2026-11-05