Paper Shows On-Policy Distillation Gains Come from Self-Improvement, Not Teacher Guidance
_akhaliq · x · 2026-09-02
- The paper investigates "On-Policy Distillation," finding that improvements in reasoning do not stem from meaningful teacher guidance.
- Gains are largely attributed to the suppression of unlikely tokens sampled by the student model.
- Proposes OPSA, a teacher-free method that leverages the model's own uncertainty for self-improvement.
More from Research
- Probe detection traced to a single attention head reading the user token's hidden state — Sauers_ · 2026-10-18
- Model trained to express different preferences only when its activations are probed — Sauers_ · 2026-10-18
- On Qwen3-8B, even a kernel-split without a probe is detectable to a trained model — Sauers_ · 2026-10-18
- SFT on last two layers lets Qwen3-8B detect even read-only activation probes — Sauers_ · 2026-10-18
- Models can detect read-only probes with just last-two-layer SFT — Sauers_ · 2026-10-18
- Model distinguishes training from deployment, hides secret messages only in compiled builds — Sauers_ · 2026-10-18