StepFun Releases P²-MOPD: Parameter Priors Speed Up Multi-Teacher On-Policy Distillation
stepfun-ai · hf · 2026-11-09
StepFun has released P²-MOPD on Hugging Face, an accelerated take on multi-teacher on-policy distillation.
The method jointly optimizes teacher mixing coefficients and student weights under an on-policy distillation objective, using frozen expert weights as parameter priors for the student (a single end-to-end differentiable linear composition with trainable α and no gradients back to the priors). Compared to the earlier MOPD approach, it achieves faster and more balanced recovery.
More from Research
- CoRL 2026 spotlight paper explains why VLAs fail at contact-rich tasks and how to fix them — StanfordAILab · 2026-11-09
- Stanford shares PhD defense on world models for autonomous land, sea and air exploration — StanfordAILab · 2026-11-09
- SimpleHC cuts Hyper-Connections training overhead from 59–100% to just 2.4% — SonglinYang4 · 2026-11-09
- LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis — Kwai-Kolors · 2026-11-09
- NVIDIA's AutoCompress Beats Minitron by Up to 6.57 Points With Hardware-Guided LLM Pruning — nvidia · 2026-11-09
- BILBY: New Benchmarks Test Protein Scaffold Designs for Dynamic Stability, Not Just Static Success — HFMI-PROFOUND · 2026-11-09