New Theory Paper Shows Sequence-Level On-Policy Distillation Has a Future Agreement Bias

kalyan_kpl · x · 2026-11-20

An arXiv paper theoretically analyzes on-policy distillation (OPD) objectives, answering which tokens get prioritized, in what order, and how efficiently.

Key findings:

For teams using OPD for distillation or post-training, the paper provides principled guidance on choosing between sequence- vs token-level objectives and which divergence to minimize.

Related event: Theory Breaks Down Sequence- vs Token-Level Objectives in On-Policy Distillation(2 posts)→

Original post →

More from Research

Research channel →