Study Finds On-Policy Distillation Transfers Reasoning, Not Answers

teortaxesTex · x · 2026-08-30

This paper investigates the generalization behavior of On-Policy Distillation (OPD). Key findings include:

• Training difficulty has little effect: Problems the teacher never solves still provide useful supervision, suggesting OPD transfers the teacher's reasoning behavior rather than specific answers.
• Model origin matters: Same-origin teacher-student pairs generalize well across languages, reasoning horizons, and domains, while cross-origin teachers often transfer less effectively.
• Multi-teacher trade-offs: Routing prompts to domain experts fails to isolate influence; changing the teacher mixture causes a seesaw effect in student capabilities rather than simple skill combination.

Related event: Study: On-Policy Distillation Transfers Reasoning Behavior, Lineage Matters More Than Teacher Scores(3 posts)→

Original post →

More from Research

• New paper: agents must "forget" to keep improving in days-long autoresearch runs — matei_zaharia · 2026-10-16
• Ex-OpenAI interpretability lead: most neurons in trillion-parameter models are now legible — burny_tech · 2026-10-16
• SHELDON harness models separate hypothesis and observation spaces for scientific agents — shi_weiyan · 2026-10-16
• WEDGE turns navigation agents' own failures into targeted training worlds — mohitban47 · 2026-10-16
• Researchers propose publicly logging OpenReview desk rejections with submission bans — ChenhaoTan · 2026-10-16
• MIT's Buehler on AI as a new language of science: superintelligent partners and executable papers — CatAstro_Piyush · 2026-10-16

Research channel →