Microsoft’s ReOPD reuses teacher prefixes to make agent distillation 4× faster
dair_ai · x · 2026-07-27
ReOPD reuses teacher trajectories to cut agent-training cost
Microsoft Research and the University of Amsterdam propose Replayed-Prefix On-Policy Distillation (ReOPD) for agentic tasks.
- Instead of fresh environment rollouts on every update, ReOPD reuses pre-collected teacher trajectories as replayed prefixes.
- The student acts at selected steps while the teacher provides dense per-step supervision without new environment interaction.
- The paper identifies a “prefix trap”: making histories more on-policy for the student can push the teacher onto states where its targets become unreliable.
- ReOPD treats this as reliability-aware prefix distribution design and uses a step-decaying sampling schedule that emphasizes earlier, lower-shift prefixes.
- Across math reasoning with Python and search environments, it matches or improves accuracy, uses zero tool calls during student training, and runs at least 4× faster per rollout than standard OPD.
More from coding & agent
- Victor Taelin updates his agent memory system with tree expansion and long-term recall — AccBalanced · 2026-07-27
- One developer merges Claude Code and Codex histories into a 10-million-message archive — doodlestein · 2026-07-27
- Google roundup lists 15 free AI tools across marketing, coding, docs and music — aigclink · 2026-07-27
- A 9B Ollama agent can run a fully local DJ radio with tools, memory, and TTS — pinku1 · 2026-07-27
- Bugbot rejects an MCP permission flag because it would break path-scoped isolation — zeeg · 2026-07-27
- One GPT-5.6 agent is guarding a Blink security system while another makes a parody rap album — repligate · 2026-07-27