Harness-Aware Distillation Boosts Small-Model Agent Harness Use 65.7% to 73.1% on ALFWorld
omarsar0 · x · 2026-10-07
A new paper introduces Harness-Aware Distillation for small language model agents: query the same teacher with and without harness information, train the student only on cases where its action changes, and filter pairs contradicting harness records—no task rewards or success labels needed.
Key findings:
- Adding harness info to on-policy distillation raises the student's harness-information usage on ALFWorld from 65.7% to 73.1%, but success stays flat (43.1% → 43.5%)
- The student reaches 63.4% on unseen ALFWorld tasks
A practical, reward-free distillation recipe for teams deploying small-model agents.
More from Research
- COSMI composes single-object captures into 222k multi-object interaction sequences, 30x larger than prior sets — UniTuebingen · 2026-10-07
- Learned latent protein languages cut structure-prediction perplexity 34% and run ~1000x faster than AlphaFold2 — Mahdi Pourmirzaei · 2026-10-07
- UWaterloo's IGMBench tests world editing in Minecraft and Terraria; best agent solves 78.2% of tasks — UWaterloo · 2026-10-07
- Polar probe paper accepted at COLM 2026: linearly decoding semantic geometry in LLMs — JeanRemiKing · 2026-10-07
- AI math proofs shift to open-source-style collaboration, square packing shows — ctjlewis · 2026-10-07
- Overmind: Open Platform That Turns Production Traces Into Fine-Tuning Data for Agents — cneuralnetwork · 2026-10-07