Tsinghua team: 16 diverse queries match full-data on-policy distillation
青稞AI · wechat · 2026-09-04
A Tsinghua team released OneShotOPD, the second paper in the RethinkingOPD series, asking why on-policy distillation (OPD) keeps improving even on a single sample:
- Data side: OPD trains on states visited during on-policy rollouts, not the queries themselves. One query with N rollouts per step yields many teacher-supervised states. State coverage: 1 query covers 71.5% of the full-data state space; 16 semantically diverse queries reach 98.9%, matching full-data OPD.
- Algorithm side: The bottleneck is the absorption rate — the fraction of remaining teacher-student gap eliminated per update steadily declines, at nearly the same rate whether training on 1, 4, 16, or all 17k queries. OPD is "data-saturated but algorithm-starved."
- Extreme tests: On MOPD, 16 queries per domain match full data. Empty templates (only a <think> tag) and WildChat data (only 0.17% math-related) both approach the math real-data baseline with one-third to one-half the rollout tokens.
Takeaway: input data acts mainly as a state generator — anything that boots the student's reasoning into teacher-supervised territory works. Paper and code are open-sourced.
Related event: Tsinghua's One-Shot OPD Matches Full Distillation with 16 Queries(2 posts)→
More from Research
- A real larva's connectome — 2,675 cells, 14,066 connections — now swims in a 3D simulation — eptwts · 2026-10-21
- Stanford's T2-z lets robots probe hidden physics with a poke, lifting success from 34% to 73% — stanfordnlp · 2026-10-21
- Fei-Fei Li team unveils FACT, a force-aware foundation model for robot dexterity — stanfordnlp · 2026-10-21
- Upcoming paper claims big models can handle latency-critical real-time robotics — ManlingLi_ · 2026-10-21
- NVIDIA open-sources Flora, a robotics data engine 125x cheaper than GPT-6 Astra — DrJimFan · 2026-10-21
- openRxiv explains its approach to AI amid growing researcher adoption — anshulkundaje · 2026-10-21