OPD² distills only the reasoning gains added by tuning, not the teacher’s old style
coallaoh · x · 2026-07-23
- The post highlights On-Policy Delta Distillation (OPD²), a new distillation method for reasoning models.
- Unlike standard on-policy distillation, which trains the student to imitate the teacher’s full output distribution, OPD² focuses on the delta signal: the difference between the teacher and its base model after reasoning tuning.
- The goal is to distill only the capability added by reasoning tuning, rather than also copying the teacher’s pre-existing style and preferences.
- The attached figure contrasts the two setups and frames OPD² as a way to preserve the learning trajectory that produces reasoning improvements.
More from Research
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11
- Navier-Stokes, Riemann, P vs NP: what this week's math buzzwords mean for you — koltregaskes · 2026-09-11
- Fruit fly brain as an LLM: connectome-driven language model demo goes live — ngxson · 2026-09-11
- Harry Collins: LLMs can't do frontier science because they can't invent new language — whoamisri · 2026-09-11
- The Waymo effect: how AI is quietly making research less collaborative — JohnHammersley · 2026-09-11
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11