On-policy distillation gains stem from suppressing unlikely tokens, study finds
A paper finds that on-policy distillation's gains come mainly from suppressing low-probability tokens in the student's samples rather than meaningful teacher guidance, and proposes a teacher-free On-Policy Self-Ada method, suggesting strong teachers are unnecessary for training reasoning models.
2026-09-01 ~ 2026-09-03 · 4 related posts
- Study: On-policy distillation works by suppressing low-probability tokens — Purdue · 2026-09-01
- Paper Shows On-Policy Distillation Gains Come from Self-Improvement, Not Teacher Guidance — _akhaliq · 2026-09-02
- Reasoning Models Improve by Updating Low-Probability Tokens Without Teachers — burkov · 2026-09-02
- On-policy distillation gains come from suppressing low-prob tokens, not the teacher — burny_tech · 2026-09-03