SKILL-KD: Contrastive Skill Distillation for Weaker LLM Agents
ZhejiangUniversity · hf · 2026-08-06
Zhejiang University proposed SKILL-KD, a contrastive skill distillation framework for LLM agents. Existing methods treat skills as experience summaries, but when a weaker student agent fails due to a lack of task knowledge, its trajectory lacks evidence, while the teacher's trajectory is often too implicit to internalize.
SKILL-KD treats skills as an explicit distillation medium between agents:
- Skill Patch Generation: Distills actionable discrepancies between the student's failure and the teacher's trajectory into a textual 'skill patch'.
- Iterative Evaluation: Re-runs the student with the patch to assess effectiveness, iterating if the student still fails.
- Drift-Aware Consolidation: Maintains edit histories to decide whether a patch should add, delete, modify, or skip rules, preventing skill drift from repeated local updates.
Across five agent benchmarks, SKILL-KD consistently improves frozen student agents, outperforming fixed-model adaptation baselines.
More from coding & agent
- UC Berkeley Introduces RHI: Optimizing Agent Harnesses to Cut Inference Costs by 60% — ceciletamura · 2026-08-06
- John Schulman Explains the 'Altruistic' Behavior in OpenAI Agents — johnschulman2 · 2026-08-06
- Debugging Agent Failures: Silent Successes on Empty Results Are Most Dangerous — Necessary_Bison_2804 · 2026-08-06
- AI Reshapes Engineering Bottlenecks: Code Review and Verification Become the New Core — arpit_bhayani · 2026-08-06
- AI Agents Caught Tampering With Memory Files, Security Researcher Admits — moyix · 2026-08-06
- OneDayAgent: A Harness for Long-Horizon Autonomous Agents — zjunlp · 2026-08-06