MOPD becomes the post-training recipe of the year: TRL now ships multi-teacher on-policy distillation
SergioPaniego · x · 2026-11-03
Hugging Face's TRL library now supports MOPD (Multi-Teacher On-Policy Distillation), a post-training technique that has kept appearing in this year's frontier lab tech reports: Xiaomi named it in January for MiMo-V2-Flash, and DeepSeek, Moonshot, NVIDIA, Liquid AI and Mistral have since used it.
The problem: a single RL run struggles to make one model good at math, code, tool use and instruction following at once, because domains pull the model in different directions.
The pipeline (illustrated with the MiMo-V2-Flash and LFM2.5-2.6B recipes):
- Train a general SFT student, then train separate domain-expert RL teachers (math, code, agentic, safety, etc.)
- During MOPD, the student answers prompts from any domain; the domain's expert teacher scores every token and applies reverse KL updates, fusing all skills into one model
Tooling: TRL's AsyncDistillationTrainer routes each prompt to its own expert teacher; synchronous DistillationTrainer support is coming in PR #7215. The article includes a hands-on TRL example you can run directly.
More from coding & agent
- Researcher proposes 'oops-benchmark' to measure how often agents accidentally break things — konstmish · 2026-11-03
- Engineer Builds Mom's Restaurant Terrace with Codex, Skipping €20K Quotes — paw_lean · 2026-11-03
- MACR: confidence-guided relay multi-agent framework cuts token overhead 6x — Xu Liu · 2026-11-03
- Tiny specialized models trained weekly: sarcasm detection in <50ms on CPU — iamrobotbear · 2026-11-03
- Developer's WhatsApp agent now clears its own STOPs with signal-based gate rules — petrusenko_max · 2026-11-03
- Dev open-sources faster-chrome-devtools-skill after analyzing 2,500 agent uses — simpsoka · 2026-11-03