Countering LeCun: LLMs Store Shapes, Not Facts, and Do Solve New Problems
teortaxesTex · x · 2026-09-25
Responding to Yann LeCun's claim that scaling LLMs can't reach AGI because they're 'gigantic memory and retrieval systems,' the author argues that von Neumann fit the same description yet was creative. LLM weights encode the shape of representations, not stored facts — facts are recomputed on the fly, even in engram-style memory modules. This is why LLMs aren't slavishly bound to training data and can genuinely find solutions to new problems.
More from AGI Musings
- Vishal Misra: Watchmen's Ozymandias is the prototype of effective altruism thinking — vishalmisra · 2026-09-25
- YTL AI Labs CEO: Headcount Unchanged After AI-Generated Coding, Judgment Beats Typing — cen6wkf · 2026-09-25
- Anti-AI discourse fails to stick in Singapore the way it does in the US — menhguin · 2026-09-25
- Opinion: Agent swarms are for finding patterns in data, not chat apps — eyishazyer · 2026-09-25
- Realtime multimodal models plus steering could yield qualitatively different agents — jakedahn · 2026-09-25
- NTT researcher on BS TBS: AI may be entering an RSI loop, alignment needs society-wide debate — Hidenori8Tanaka · 2026-09-25