Founder pushes back on LeCun: autoregressive LLMs can backtrack and search too
john__allard · x · 2026-11-23
johnallard responds to Yann LeCun's claim that autoregressive LLMs have a fundamental flaw fixed only by backtracking, search or tool use. He argues a model can sample 'wait, that's not right' and switch paths while still generating one token at a time — semantic backtracking within autoregression, rewardable by RLVR even without tools. With tools, sampled calls return into context and generation continues. He asks what redesign ever made LMs cease to be autoregressive, after LeCun cited FAIR's Toolformer (Jan 2023).
More from AGI Musings
- Physicist Anthony Aguirre publicly challenges AGI lab staff: you're building superintelligence that makes humans cognitively obsolete — michael_nielsen · 2026-11-23
- Oxford researcher pushes back on "alien mind" AI proofs: codebases are unreadable too — StefanoGogioso · 2026-11-23
- Healthcare AI advocate to doctors: AI changes the workflow, not the mission of medicine — HealthcareAIGuy · 2026-11-23
- Agents are crushing through a decade of neglected internet-facing attack surface — WeldPond · 2026-11-23
- 'Billion parameters' fuels AI hype — proposal to adopt GiP unit like storage — tennisanybody · 2026-11-23
- Geoffrey Hinton: we're making alien beings, not computer software — haider1 · 2026-11-23