STOPD cuts on-policy distillation rollout tokens by 25–48% with unbiased random truncation, 2.2x speedup
机器之心 · wechat · 2026-11-19
Researchers from Cornell and Georgia Tech propose STOPD to slash the cost of on-policy distillation (OPD), where each training round requires full student rollouts plus per-token teacher feedback.
Method: sample a stopping position before generation, truncate rollouts early, and compensate sampling bias via inverse-probability weighting; block truncation (default width 128) caps tail weights from 16,384 down to 128. They prove the weighted gradient sum matches full rollouts in expectation, with per-step error bounds for SGD, Adam, and Muon.
Results (Qwen3 teachers, 1.7B–8B students): math speeds up 1.8x with 34% fewer tokens, code 2.2x with 48% fewer, instruction-following 1.3x with 25% fewer — accuracy on par with vanilla OPD throughout. Larger batches damp loss jitter (12.2%→6.0% at batch 512). Unlike fixed-length truncation (ESR), STOPD preserves supervision on late tokens including natural EOS, keeping stopping behavior healthy.
Paper and code are open-sourced; the authors see applications to other token-level supervised training like SFT and offline distillation.
More from Research
- tla-kit: the full TLA+ toolchain in one reproducible container, plus a 14-tool MCP server — Scottie-2222 · 2026-11-19
- Free University-Level Course Dives Into the Math of Transformers and Token Dynamics — CatAstro_Piyush · 2026-11-19
- Anshul Kundaje: research artifacts should be version-controlled, not frozen as a single 'final version' — anshulkundaje · 2026-11-19
- META-Robot from HKUST completes 44/56 real-robot trials with self-repair — jiqizhixin · 2026-11-19
- Sebastian Raschka Adds Appendix on Positional Embeddings, Including YaRN and ALiBi — rasbt · 2026-11-19
- Ant Lingbo embodied AI contest: 15 teams, one shared VLA model, 48-hour real-robot showdown — 机器之心 · 2026-11-19