CriticOPD: Teacher Critiques Fix On-Policy Distillation, +4.57 Points on Qwen3-1.7B
tw_killian · x · 2026-10-17
- Key finding: In on-policy distillation (OPD), teacher models often follow the student's mistakes when continuing a flawed solution—even on problems they can solve from scratch, making supervision unreliable.
- Method: CriticOPD has the teacher critique incorrect student rollouts using the reference answer; the student rewrites the continuation from that feedback. The teacher scores the rewritten suffix with feedback in context, while the student learns from it without seeing the feedback.
- Results: Across six math reasoning benchmarks, average accuracy improves by 4.57 points on Qwen3-1.7B and 4.06 points on Qwen3-0.6B over OPD, with the largest gains on competition problems.
- Paper, code, and project page are publicly available on Hugging Face and GitHub.
More from Models
- Free DeepSeek 4.1 and GLM-5.3 Flash on OpenCode rate-limit after 5.2M/800k tokens — FanaHOVA · 2026-10-17
- Epoch AI's SpireClimb: models beat human author at Slay the Spire but still err each act — Jsevillamol · 2026-10-17
- Qwen3.8 Flash IQ3_S hits 40-50 tok/s at 131k ctx on 12GB VRAM + NVMe — bodhi371 · 2026-10-17
- Kimi K3 three months later: 1.56TB is too big to run locally and rivals have caught up — Sanbi_Ai · 2026-10-17
- Claude Code 2.1.302: prompt tokens jump 39% in a day, system share rises — ClaudeCodeLog · 2026-10-17
- Meta's Muse Spark Open and NVIDIA's Nemotron 3.5 lag the frontier on 3D building bench — ZhitingHu · 2026-10-17