Distillation Debate: Frontier CoTs Too Off-Policy For Tiny Models, 27B Is The Better Teacher
JoshPurtell · x · 2026-09-03
In an X debate over whether a major lab distilled another model, JoshPurtell breaks distillation into two paths: (1) jailbreaking APIs to copy raw chain-of-thought — highly suspicious and likely to get accounts banned; (2) running the model on dev tasks and training on tool calls/non-reasoning outputs — far harder to detect.
His core claim: frontier reasoning traces distill poorly into tiny models like Qwen 3.5 0.8B because they're too off-policy (too smart/terse); a 27B teacher is the better fit. He adds that single-shot simple tasks like Shopify's don't benefit much from copying frontier outputs, while long-horizon and coding tasks might. Counterparty VivaLaPanda pushes back, noting that distilling a big general model into a smaller task model is standard practice across the industry.
More from coding & agent
- Anthropic borrows lawyers' ethics for Claude agents that closed 186 deals with no human approval — rohanpaul_ai · 2026-10-18
- Prompt injection via email can hijack your AI agent — jev-guard adds risk-scored guards — sujingshen · 2026-10-18
- 14-Agent AI Film Pipeline on One RTX 5090: Fixing Cut Defects Between Clips — developervkmp · 2026-10-18
- Meta paper introduces "agent plasticity": best-performing agent isn't the most efficient learner — rohanpaul_ai · 2026-10-18
- Dev ships a bushfire prediction platform in a weekend with frontier models — CtrlAltDwayne · 2026-10-18
- 10 open-source GitHub repos that let AI agents control your phone, from Open-AutoGLM to mobile-use — victor_explore · 2026-10-18