Order-Only On-Policy Distillation Matches Token Rankings to Boost Reasoning Without Temperature Tuning
Guangzeng Han · hf · 2026-11-04
A new distillation method, Order-Only On-Policy Distillation (O³PD), trains student models to match the teacher's token rankings rather than their probability values.
- Compared to standard OPD (matching probability values) and SFT (imitating a single choice), O³PD only aligns the teacher's preference order (e.g., A>B>C)
- This improves student reasoning performance without any temperature tuning
More from Research
- ParallACT benchmark shows multi-agent communication barely helps: 16%→32% at 2.23x token cost — EliasEskin · 2026-11-04
- DR Pixel: open-source page-image deep research agent, 27B model matches GPT-5.4 — sewon__min · 2026-11-04
- FlexGlue: open-source pixel-perfect image matching that rejects doppelganger pairs — ducha_aiki · 2026-11-04
- Formally verifying RISC-V CPU mul/div instructions with one AI theorem-prover prompt — satnam6502 · 2026-11-04
- Your analytics agent is summing a 100-row page and calling it the total: five tool-design fixes from Cube's production harness — AbleBranch6 · 2026-11-04
- WhatIf-Bench exposes extrapolation gap in time-series foundation models, skill drops to -7% — irinarish · 2026-11-04