DeepMind Paper Shows Sequence-Level Supervised Losses for Autoregressive LLMs Are Fundamentally Flawed
mblondel_ml · x · 2026-11-17
A new arXiv paper by Mathieu Blondel et al. (Google DeepMind) analyzes sequence-level supervised learning for autoregressive LLMs.
- Standard token-level cross-entropy under teacher forcing suffers from exposure bias; the authors ask whether sequence-level losses incorporating inference-time decoding (greedy decoding or autoregressive sampling) can work instead.
- Two natural attempts are proven fundamentally flawed: the losses are unbounded below. Using autoregressive Gumbel tricks, the sampling-based loss is shown to be the expected greedy loss under Gumbel perturbations.
- The pathology stems from the decoded sequence depending on model parameters. Applying a stop gradient to the sampling procedure yields a moment-matching direction that, while not the gradient of any loss, provably defines a valid descent direction in function space for the autoregressive negative log-likelihood.
More from Research
- Zuse: decision models for 27 European languages hit 87.5% accuracy at 27B — bineric · 2026-11-17
- Copenhagen postdoc chance at biomaterials × machine intelligence, deadline Dec 6 — _onionesque · 2026-11-17
- $1.1M Templeton grant funds Global Virtues Atlas of moral beliefs for AI alignment — MohammadAtari90 · 2026-11-17
- Dev proposes swarm of specialist diffusion models for code where invalid syntax is unrepresentable — SubtleGradient · 2026-11-17
- Economists warn peer review is collapsing under 99% AI-generated papers and referee reports — Afinetheorem · 2026-11-17
- Meta's Self-Improving Memory: agents discover their own memory skills and distill them into weights — jaseweston · 2026-11-17