Theoretical analysis clarifies how on-policy distillation objectives shape what student LLMs learn

heghbalz · x · 2026-11-20

Noam Razin shared a theoretical thread analyzing on-policy distillation (OPD), now a core part of LM post-training. With many OPD objectives available, practitioners lack clear guidance on which to choose. The work theoretically examines how this choice affects what the student model learns and how fast, comparing sequence-level vs token-level objectives and Forward vs Reverse KL divergences.

Related event: Theory Breaks Down Sequence- vs Token-Level Objectives in On-Policy Distillation(2 posts)→

Original post →

More from Research

Research channel →