Waypoint-OPD: Alibaba Gates On-Policy Distillation via Early-Late Teacher Shift Agreement
Bingxi Hou · hf · 2026-10-16
Alibaba Cloud researchers propose Waypoint-OPD, an improvement over policy-shift distillation. Standard approaches use the log-probability difference of a teacher before and after RL as supervision on student-generated trajectories, but this endpoint shift cannot reveal whether local action preferences evolved consistently during teacher training—early and late shifts can reinforce or partially cancel each other.
Waypoint-OPD uses an intermediate teacher checkpoint from the same RL run to compare early and late shifts in relative action preferences at student-visited prefixes, gating the endpoint shift by directional agreement and suppressing it when shifts point in opposing directions. Compared to endpoint-only transfer, it requires just one extra frozen-teacher scoring pass, improving reasoning on students like Qwen3-8B/14B/30B distilled from Qwen3-4B teachers with minimal overhead.
More from Research
- Is math just a language? A debate on why LLMs are good at mathematics — LucaAmb · 2026-10-16
- Alibaba open-sources OpenGCUA, a full-stack generalist computer-use agent; +19.6 points on OSWorld-v2 across 8 models — Tongyi-MAI · 2026-10-16
- CoDA co-evolves simulation and robot policies via shared failure memory, lifting OOD success from 5.3% to 27.6% — PLAN-Lab · 2026-10-16
- Cambridge NLIP seminar: Danae Sanchez Villegas on multimodal reasoning beyond final answers — nikaletras · 2026-10-16
- DKGI paper: agents write their own tools to search knowledge graphs, beating baselines by 6.7 Hit@1 — yuz9yuz · 2026-10-16
- ETH Zurich's LFD lets a frozen LLM teach drug-design generators natural-language goals — rohanpaul_ai · 2026-10-16