ByteDance study finds on-policy distillation leaves models 2.45-16.25% below base on out-of-distribution tasks
ByteDance · hf · 2026-11-05
A ByteDance-led paper (with UChicago, Duke, UIUC, Northwestern) systematically examines the generalization limits of on-policy distillation (OPD) and multi-teacher OPD (MOPD) for post-RL capability consolidation.
Key findings
- Despite near-perfect in-distribution scores (96.04-102.73% of teachers' IND performance), OOD scores remain 2.45-16.25% below the base model, showing teachers' generalization is not fully transferred.
- Across three compositional settings on math reasoning, coding, agent, and long-context tasks with Qwen3 models, MOPD's OOD performance on composed tasks trails the best teachers by 18.19-24.34%.
- Update geometry analysis: distillation updates are smaller and more spectrally concentrated than RL baselines, with more shared structure under MOPD.
Mitigation
The authors propose geometry-based patching and base-preserving replay, improving OOD performance while largely preserving IND scores; jointly addressing teacher capability and data coverage bottlenecks eliminates OOD forgetting. The work highlights a blind spot in (M)OPD-based consolidation and the need for algorithms explicitly supporting OOD retention.
More from Research
- GitHub Next's PDD Kit uses formal methods to prove AI-generated web app code is correct — 0xkarasy · 2026-11-05
- Meta's JiTTesting deploys LLM-generated tests to catch bugs pre-merge, cutting human review 70% — AxSaucedo · 2026-11-05
- 演示显示 LLM 能在普通散文里藏密码,人类肉眼难察觉 — dosco · 2026-11-05
- Robot benchmarks slammed as fantasy: real robots can't afford $10k GPUs for VLA models — AIFlow_ML · 2026-11-05
- Agents ace isolated tasks but drop to ~38% at maintaining codebases over time — FullStackFrenzy · 2026-11-05
- RoboColosseum founder: robotics lags AI because it lacks real-world benchmarks — DJiafei · 2026-11-05