On-policy distillation improves for hundreds of steps from a single query — it's algorithm-starved, not data-starved

Thinking-Space · hf · 2026-09-04

"Rethinking On-Policy Distillation of LLMs II" reports a counterintuitive finding.

• Core result: with just one training example (a single query), on-policy distillation keeps improving the student model over hundreds of training steps
• Mechanism: the student rapidly covers the teacher's states, so data is not the bottleneck
• However, student-teacher alignment remains slow — the authors conclude the method is "algorithm-starved" rather than data-starved, pointing future gains toward algorithm design rather than more data
• Directly informs how to allocate effort (data cost vs algorithmic innovation) in distillation training

Original post →

More from Research

• FlashRender: Few-step camera-controlled generative rendering via MeanFlow distillation — everex · 2026-09-04
• LatentStream: progressive latent memory evolution for streaming video understanding — Hongyu Qu · 2026-09-04
• Temporal Context Routing aligns script timing in joint audio-video generation — Yichen Liu · 2026-09-04
• Puffin-World scales unified multimodal model with native 3D world states — Kang Liao · 2026-09-04
• Agent reliability degrades with trajectory length, but no benchmark isolates it, dev finds — rio_ARC · 2026-09-04
• The Bayes Bandit: A Mathematical Take on Curiosity in Reinforcement Learning — CatAstro_Piyush · 2026-09-04

Research channel →