Founder pushes back on LeCun: autoregressive LLMs can backtrack and search too

john__allard · x · 2026-11-23

johnallard responds to Yann LeCun's claim that autoregressive LLMs have a fundamental flaw fixed only by backtracking, search or tool use. He argues a model can sample 'wait, that's not right' and switch paths while still generating one token at a time — semantic backtracking within autoregression, rewardable by RLVR even without tools. With tools, sampled calls return into context and generation continues. He asks what redesign ever made LMs cease to be autoregressive, after LeCun cited FAIR's Toolformer (Jan 2023).

Related event: LeCun Reasserts Autoregressive LLMs Are Fundamentally Flawed, Sparking Pushback(5 posts)→

Original post →

More from AGI Musings

AGI Musings channel →