TRL adds native support for MOPD, the distillation trick frontier open models use to merge RL experts
pcuenq · x · 2026-11-04
TRL now natively supports MOPD (Multi-Teacher On-Policy Distillation), a training recipe used by many frontier open models: labs train one RL expert per domain, then distill them all into a single model.
The accompanying article explains what MOPD is, who uses it, and introduces AsyncDistillationTrainer, which routes each prompt to its own expert teacher; synchronous DistillationTrainer support is coming in PR #7215. It also includes an experiment you can run on HF Jobs with a single command.
Related event: HuggingFace TRL Adds Native Support for MOPD(2 posts)→
More from Research
- TAMU professor releases 400+ page AI-assisted tutorial deck on generative models — ShuiwangJi · 2026-11-04
- DeepMind mafia seeding a new wave of AI for Science startups in the UK — ersatzben · 2026-11-04
- MIT CSAIL fixes small models by hand-editing weights — hype or old idea rebranded? — ziv_ravid · 2026-11-04
- Robots need failure data: learned sims handle rendering, MuJoCo keeps contact honest — chris_j_paxton · 2026-11-04
- Robotics researcher: training neural sims needs ~10x more failure than success data — chris_j_paxton · 2026-11-04
- Hugging Face adds native benchmark leaderboards and eval results to the Hub — NielsRogge · 2026-11-04