SEED: Self-Evolving Agent Distillation
Jinyang Wu · hf · 2026-07-17
## SEED: Self-Evolving On-Policy Distillation This paper proposes **SEED (Self-Evolving On-Policy Distillation)**, aiming to bridge the supervision gap between "trajectory-level sparse rewards" and "token-level policy learning". The authors convert completed on-policy trajectories into reusable natural language "skills", then distill the behavioral differences brought by these skills back into the policy model. This forms a self-evolving closed loop where the policy improves its trajectory summarization capabilities while updating. ### Core Approach - The policy first analyzes completed trajectories to extract reusable workflows, key observations, and failure avoidance rules - During RL, the current policy handles both sampling and extracting hindsight skills from trajectories - The action probability difference between "standard context" and "skill-enhanced context" is converted into a dense, token-level on-policy distillation signal - This signal is jointly optimized with outcome-based RL, keeping auxiliary supervision aligned with the current trajectory distribution ### Results - Brings consistent improvements on both text-based and vision-based agent tasks - Enhances both performance and sample efficiency - Shows strong generalization to unseen scenarios Code is open-sourced.
Related event: SEED: Self-Evolving Distillation for Agentic RL(2 posts)→
More from coding & agent
- Matt Pocock says every new codebase turns legacy within days — mattpocockuk · 2026-07-21
- Ramp says its model router cuts LLM costs by 30% by picking the cheapest good model — rohanpaul_ai · 2026-07-21
- Meta and Unity link AI workflows to Quest development across setup, input and validation — Vjeux · 2026-07-21
- AI Engineer World’s Fair spotlights Kids Day with 87 children learning to code — steveonjava · 2026-07-21
- Looking Glass adds persistent coding sessions that can schedule their own next turns — teleport66 · 2026-07-21
- Kimi K3 looks stronger and about 5× cheaper on a frontend dashboard task — OwariDa · 2026-07-21