Data-Efficient Agent Distillation: Small Model Matches 9x Larger Teacher with Only 19 Samples
ctnzr · x · 2026-08-06
Agent distillation can be surprisingly data-efficient. By using trajectory data from only 19 problems, researchers distilled Inkling-Small into Nemotron-3-Nano-30B-A3B via LoRA SFT.
The small model successfully matched the performance of its teacher model—which is nine times its size—on Kubernetes incident-diagnosis tasks. This is highly encouraging for post-training task-specific models in data-poor regimes.
More from coding & agent
- Continual Learning Bench: Simple Context Memory Beats Expensive Dedicated Systems — ajratner · 2026-08-06
- Dev Advocates Ditching Claude for GPT or Kimi in Coding — MarcJSchmidt · 2026-08-06
- AI Toolkit Helper Released: Utility Tools for Model Trainer — ostrisai · 2026-08-06
- DeepSeek API Adds Responses Format with Built-in Web Search and Codex Support — teortaxesTex · 2026-08-06
- Shopify's Continual Learning Flywheel Beats Frontier Models, Cuts Costs 96% — MParakhin · 2026-08-06
- Don't Use Prompts to Govern Agents: Enforcement Belongs at the Tool Boundary — No-Conflict4823 · 2026-08-06