Practitioner finds multi-teacher policy distillation fails to beat plain GRPO so far

zainhas · x · 2026-11-16

AI engineer zainhas reports that in his policy distillation experiments, multi-teacher setups haven't significantly outperformed simply running GRPO on specialized datasets. He suspects insufficient teachers/domains, data being too easy, or unknown factors, and shares a figure of a three-stage MOPD pipeline (SFT → per-domain expert teachers → Reverse KL distillation into a student model), soliciting discussion.

Original post →

More from Research

Research channel →