On-policy distillation improves for hundreds of steps from a single query — it's algorithm-starved, not data-starved
Thinking-Space · hf · 2026-09-04
"Rethinking On-Policy Distillation of LLMs II" reports a counterintuitive finding.
- Core result: with just one training example (a single query), on-policy distillation keeps improving the student model over hundreds of training steps
- Mechanism: the student rapidly covers the teacher's states, so data is not the bottleneck
- However, student-teacher alignment remains slow — the authors conclude the method is "algorithm-starved" rather than data-starved, pointing future gains toward algorithm design rather than more data
- Directly informs how to allocate effort (data cost vs algorithmic innovation) in distillation training
Related event: One training example recovers 72% of on-policy distillation gains(2 posts)→
More from Research
- Kaggle launches Google Quantum AI verifiable quantum circuit compilation competition — JFPuget · 2026-10-23
- Aggregating lab tokenizers reveals ~994 tokens stable across Google, DeepSeek, Meta, Anthropic — kalomaze · 2026-10-23
- Researcher uses Grok to hunt LLM-driven UAE bot network on X with self-written prompts — FlorianGallwitz · 2026-10-23
- Dev converts Qwen3-0.6B into a MoE model: quality drops and no speed gain at this scale — Bayliner1980 · 2026-10-23
- Tencent's WeChat Vision releases WeVisOne-VL, unifying object- and pixel-level perception — jacek2023 · 2026-10-23
- Batching benchmark: 50x throughput for ~24% latency cost on PyTorch — cneuralnetwork · 2026-10-23