Relay-OPD improves on-policy distillation by handing failed prefixes back to the teacher
zju · hf · 2026-07-29
Relay-OPD addresses a prefix failure issue in on-policy distillation: when a student goes down the wrong reasoning path, later supervision becomes noisy and wastes compute.
- It detects failed-prefix handoff points with a label-free trigger based on teacher–student continuation asymmetry.
- During training, the teacher briefly takes over at those points to generate a relay trajectory, then the student resumes optimization on that trajectory.
- The method uses a limited relay budget to focus intervention on early critical positions while staying close to the student policy.
- With Qwen3-4B-Instruct-2507 as teacher and Qwen3-0.6B / 1.7B-Non-Thinking students, it achieves the best or second-best result on every one of 8 math reasoning benchmarks.
- Compared with standard OPD, it improves average performance by +5.73% for the 1.7B student and beats FastOPD by +1.49%; the training trajectory length is cut by over 50%.
Related event: Relay-OPD Framework Fixes Prefix Failures in On-Policy Distillation(2 posts)→
More from Research
- Transforming quadruped robot runs on real hardware: per-pose RL policies in Isaac switched on morph — _Stocko_ · 2026-10-16
- KinoGen synthesizes robot loco-manipulation reference motions directly via kinematically grounded video generation — ChongZzZhang · 2026-10-16
- Science Robotics paper introduces F³, predictive event-based vision beyond optical flow — CSProfKGD · 2026-10-16
- VideoGen-Agent: An 8B Multimodal Agent Turns Video Generation Into a Tool-Use Loop — jiqizhixin · 2026-10-16
- Comprehension Audits: New Paper Proposes a Gate for AI-Written Code at Frontier Labs — ronbodkin · 2026-10-16
- Not Boring #215: Cammarata's Clean Networks Claim, Tesla's Flying Roadster, Tau Cut 73% — Not Boring (Packy McCormick) · 2026-10-16