Exploring Peer Distillation and Post-Training Between Equal Models
A community discussion explored peer distillation, questioning how well two equally performing pre-trained models can distill each other using rollouts after post-training, a setup rarely covered in existing literature.
2026-07-18 ~ 2026-07-18 · 2 related posts
- How Effective Is Distillation From Peer Models? — maxsloef · 2026-07-18
- Comparing Peer Distillation vs Post-Training — maxsloef · 2026-07-18