Both greedy and AR-sampling sequence losses are unbounded below, thread explains

mblondel_ml · x · 2026-11-17

Part two of Blondel et al.'s paper thread: the two natural sequence-level losses based on greedy decoding and autoregressive sampling are provably unbounded below—optimizing them would make the model diverge—and are connected via an AR Gumbel trick. See the thread's first post for the full analysis.

Related event: DeepMind paper proves sequence-level supervised losses for autoregressive models can be fundamentally flawed(5 posts)→

Original post →

More from Research

Research channel →