LeCun: autoregressive LLMs have a basic flaw; giffmana pushes back that fixes stay autoregressive
giffmana · x · 2026-11-23
Yann LeCun, responding about Toolformer (the 2023 FAIR paper that popularized LLM tool calls), doubled down on his talk's claim: autoregressive LLMs have a basic flaw. You can fix it by making them no longer purely autoregressive—e.g., backtracking, search, tool use, multiple trial-and-error.
giffmana countered that all of these remedies still happen auto-regressively; only swarms constitute true parallel search. The exchange cuts to the core architecture debate: are tool calls and search genuine fixes or patches over a fundamentally limited paradigm?
More from AGI Musings
- Apple's appenz: agent swarms are filling in NVIDIA's CUDA moat — appenz · 2026-11-23
- What comes after know-how? LeCun weighs in on AI redefining intelligence — alexbilz · 2026-11-23
- METR-fit extrapolation implies 66-hour task horizons for 6.2 Astra, 128 hours for internal model — eliebakouch · 2026-11-23
- Uber burned its annual AI budget in 4 months with no visible improvement — yaroslavvb · 2026-11-23
- Post-training may exceed 75% of FLOPs, undercutting perplexity-based safety bars — stochasticchasm · 2026-11-23
- FDA greenlights first human trial of age-reversal gene therapy — Dr_Singularity · 2026-11-23