Paper: One training query recovers 72% of full-data on-policy distillation gains
heghbalz · x · 2026-09-06
A new paper, Rethinking On-Policy Distillation of LLMs II: One Training Example, probes the data-minimal limit of on-policy distillation (OPD):
- One query is enough to keep improving: one-shot OPD trains for hundreds of steps and recovers most of the full-data OPD gain across task domains and model families.
- State coverage explains it: a single query's rollouts already reach 71.5% of the states visited by full-data OPD, mostly within the first 100 steps; 16 semantically diverse queries hit 98.9% coverage and match full training.
- The bottleneck is algorithmic, not data: the student absorbs the teacher's dense token-level supervision slowly regardless of data volume. OPD is "data-overfed but algorithm-starved."
- Findings extend to multi-teacher OPD; content-light templates and off-domain queries also approach the real-query baseline.
Authors include Zhiyuan Liu and Ning Ding. Relevant to anyone paying for distillation datasets.
Related event: One training example recovers 72% of on-policy distillation gains(2 posts)→
More from Research
- JEPA may keep unpredictable features: new paper gives set-level characterization — hisspikeness · 2026-10-23
- GAFU Uses Generation-Side Representations to Boost Fine-Grained Visual Understanding in Unified Multimodal Models — Tengju Ru · 2026-10-23
- NTU Study: Learned Unmasking Policies Improve Speed-Accuracy Trade-offs in Diffusion Language Models — nanyang-technological-university-singapore · 2026-10-23
- NBER releases 'The Economics of Transformative AI' edited by Agrawal and Brynjolfsson — JMateosGarcia · 2026-10-23
- Swapping only the eval harness moves results 30%: designing agent evals like a scientist — AI Engineer · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23