On-policy supervised fine-tuning: RL-level generalization at SFT-level compute
burkov · x · 2026-10-14
- Post-training faces a dilemma: SFT is compute-efficient but generalizes poorly and risks catastrophic forgetting; RL generalizes better but needs massive compute, reliable rewards, and suffers low sample efficiency.
- The article introduces an on-policy supervised fine-tuning framework aiming to deliver RL-like generalization at standard supervised-pipeline compute costs.
- The focus is establishing the theoretical grounding for this middle path between SFT and RL.
More from Research
- Snap launches as first fully sub-quadratic LLM in production with constant compute per token — ryanallred · 2026-10-14
- KnitNet pairs generative modeling with verifier-guided optimization for RNA inverse design — chaitjo · 2026-10-14
- WOVEN benchmark finds all 38 frontier MLLMs fail visual transition reasoning far below humans — MLL-Lab · 2026-10-14
- Narrow multimodal fine-tuning induces emergent misalignment across 15 VLMs, ETH Zurich finds — Shunchang Liu · 2026-10-14
- HyperThink uses a hypernetwork to replace long chain-of-thought with one parameter update — Donggyun Kim · 2026-10-14
- Economist interviews AI agents on sudden windfalls — one goes deep on estate planning — soumitrashukla9 · 2026-10-14