KAIST paper identifies 'Teacher Drift' in multilingual reasoning distillation and teaches only when it helps
Jaewon Cho · hf · 2026-11-09
Researchers from KAIST and Korea University study Teacher Drift in on-policy self-distillation: as student errors accumulate in intermediate steps, the teacher (the same model conditioned on reference solutions) may start following the student's wrong steps instead of correcting them.
Their Teachable On-Policy Self-Distillation method detects when the teacher is still useful, distills only at teachable moments, and limits drift — improving multilingual math reasoning, especially for lower-resource languages.
More from Research
- Google's AI agents discover verified quantum programs, cutting quantum arithmetic cost 290x — dair_ai · 2026-11-09
- Survey finds continuous LM progress shifted from guidance methods to few-step generation — LucaAmb · 2026-11-09
- RoboMoCap dataset released: 200+ hours of 120Hz whole-body mocap for humanoid robot learning — DJiafei · 2026-11-09
- 5 free math textbooks that cover the core math behind machine learning, exercises included — cneuralnetwork · 2026-11-09
- Five free math textbooks that cover the math behind ML, with exercises included — cneuralnetwork · 2026-11-09
- GeoAI roundtable podcast: image classification, prompt-trained models, open tools — giswqs · 2026-11-09