1% of Tokens Can Be Enough: MBZUAI and Ant Group Improve On-Policy Distillation
jiank_uiuc · x · 2026-09-23
A MBZUAI–Ant Group paper (arXiv:2609.24432) studies gradient estimation in sparse on-policy distillation (OPD), where teacher supervision is allocated to a small subset of tokens:
- Problem: gradients estimated from sampled next tokens are noisy, so useful teacher guidance can yield noisy updates.
- Method: analyzing estimation at a fixed prefix via information geometry, the authors propose an information-efficiency ratio (IER) from a signal-to-noise decomposition, enabling token selection via a candidate-set approximation that composes with existing usefulness scores while keeping the sampled reverse-KL objective.
- Results: IER improves existing selectors on math and medical reasoning; at 0.1%–1% token budgets, sparse configurations match or exceed full OPD without selection. Gains hold in both thinking-on and thinking-off modes. Paper and code are public.
More from Research
- minpts-check: one-run predictor tells if your HDBSCAN clusters survive min_samples changes — leland_mcinnes · 2026-11-12
- Architecture study sparks monitoring concerns over a multi-hop planning spike — ChowdhuryNeil · 2026-11-12
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12