Will Brown essay: strongest model may be the wrong teacher — SFT, RL and on-policy distillation
heghbalz · x · 2026-11-17
Will Brown publishes a long-form essay 'On SFT, RL, and on-policy distillation':
- Why the SFT-then-RL pipeline works, and where on-policy distillation fits
- In on-policy distillation the student generates its own attempts and the teacher gives token-level probability feedback, so supervision covers trajectories the student actually visits, including mistakes
- Key insight: a model's problem-solving ability and its value as a training signal are different things — teacher distributions carry formatting and phrasing quirks, so same-family compatibility may matter as much as raw teacher capability
- Also covers how self-distillation goes wrong
More from coding & agent
- Cloudflare launches x402 Monetization Gateway for AI agents; Solana handles 70% of x402 txns — kleffew94 · 2026-11-17
- Mnemos x NousResearch launches agent memory engine claiming 2x gains over Claude Code — RileyRalmuto · 2026-11-17
- How Revid doubled its ChatGPT-sent signups from 355 to 781 per week in 10 weeks — tibo_maker · 2026-11-17
- FormMCP: one MCP connection to a family of desktop apps, tools loaded on demand — nodemode_ai · 2026-11-17
- Harness Adapters: RL-trained LoRA lifts Qwen3.8-27B on Pi from 29% to 63% on SWE-bench Pro — LysandreJik · 2026-11-17
- Pi harness adapter ships as a LoRA runnable on vLLM, llama.cpp, and MLX — yb2698 · 2026-11-17