NJU and JD's Road MOPD tames multi-teacher distillation gradient conflicts, gaining 5+ points
jingdong1 · hf · 2026-10-15
Nanjing University and JD.com propose Road MOPD (Cross-Teacher Guarding) to fix gradient conflicts in multi-teacher on-policy distillation (MOPD).
- Domain routing alone can't prevent conflicts: updates from all domains still hit shared parameters, so supervision effective in isolation can degrade capabilities when teachers differ in style and expertise.
- The method uses non-routed teachers as auxiliary cross-domain critics on the same student trajectories, with teacher-agreement-based token-level credit assignment and preventive multi-teacher gradient projection that adaptively controls projection strength.
- Road MOPD mitigates degradation in disadvantaged domains, gains over 5 percentage points under high conflict, and improves overall performance in low-conflict and three-domain settings.
More from Research
- Alibaba's HICO-Bench shows 36% of highly-rated agent conversations fail verification — Tongyi-ConvAI · 2026-10-15
- PixText fixes text in generated images: 83% glyph pass rate vs 37% for baselines — ATH-MaaS · 2026-10-15
- TextVerifier catches flawed Chinese glyph rendering with F1 0.844, far above OCR and VLMs — ATH-MaaS · 2026-10-15
- ByteDance's Diff Diffusion predicts code edit paths: 2.78x faster inference, 39% lower latency — Xiaolong Xu · 2026-10-15
- MimicX refines humanoid tracking with policy-in-the-loop feedback: 25.7% error cut — Shuaijun Liu · 2026-10-15
- MagicBeanStalk: 88% of RLVR data trapped in zero advantage, recursive synthesis adds 8.6 points — Lianhuiq · 2026-10-15