Study: on-policy distillation teaches new skills but not new knowledge
Yixuan Tang · hf · 2026-10-09
A controlled study asks whether on-policy distillation (OPD) gives students new factual knowledge or compositional skill for multi-step reasoning.
- Using a synthetic framework that independently controls the teacher's added facts, compositional skill, or both, across four models from three families, reverse-KL OPD reliably transfers compositional skill to unseen reasoning structures but transfers minimal factual knowledge.
- Ablating the recipe: swapping reverse KL for forward KL restores factual transfer, while student rollouts specifically improve execution of multi-step reasoning.
- Experiments on factual QA and competition math show the same asymmetry—strong reasoning gains without factual memory expansion—demonstrating OPD reorganizes existing parametric knowledge rather than expanding it.
More from Research
- Domingos Cites New Result Claiming RL Is 'Basically Useless' vs Supervised Learning — pmddomingos · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23