CriticOPD shows teachers follow student errors and fixes on-policy distillation with critiques
Kaixiang Zhao · hf · 2026-10-15
Researchers from BYU and UNC Chapel Hill identify and fix a flaw in on-policy distillation (OPD) they call teacher recoverability.
- OPD assumes the teacher can reach a correct answer from any student-written prefix; the authors show this fails on incorrect rollouts—the teacher follows student errors even though it recognizes them.
- CriticOPD has the teacher critique the wrong rollout so the student can rewrite its solution; the critique is given to the teacher as privileged information for scoring, while the student trains on the critique-free revision.
- On two Qwen3 students and six math reasoning benchmarks, CriticOPD improves over OPD, with the largest gains on competition problems.
More from Research
- Gil Kalai presents 'the most outrageous conjecture' on proving primes in sparse sets — _onionesque · 2026-10-15
- 83,017-vacancy RCT: human recruiters add no hires once algorithmic screening exists — danielrock · 2026-10-15
- RAESR Does Super-Resolution in Frozen DINOv3 Latent Space at 37ms per 512px Image — RTP-LLM · 2026-10-15
- BigCell Generates Coherent Gigapixel Whole-Slide Pathology Images, Boosting Few-Shot Classification 18% — StonyBrook-CVLab · 2026-10-15
- NavGPT-3 Reaches Human-Level Navigation: 90.43 SR on RxR-CE vs Human 90.4 — Metacognition-AI · 2026-10-15
- ByteDance Seed's Cortex Distills Context into Parametric Memory, Keeping 88.9% Gains — ByteDance-Seed · 2026-10-15