TailSFT: Microsoft Paper Shows SFT Can Wreck RL Starting Points, Gains up to 4% pass@1
rohanpaul_ai · x · 2026-09-04
- A new Microsoft + UC San Diego paper shows standard SFT can make a model look better on evals while actually being a worse starting point for RL, by wiping out rare correct behaviors RL needs to reinforce.
- TailSFT: filter out training sequences whose loss has already dropped most relative to the base model, shifting learning toward under-modeled regions of the data distribution so correct responses stay reachable under repeated sampling.
- Results: on OLMo-3 7B, pass@16 on math and coding improves up to 17% absolute with minimal overhead; subsequent GRPO runs see up to 4% absolute pass@1 gains (up to 3.93 points). The paper also introduces a lightweight diagnostic for when TailSFT helps.
Related event: Microsoft's TailSFT: Filtered Fine-Tuning Improves RL Performance(2 posts)→
More from Research
- Richard Socher: AI could compress a century of scientific breakthroughs into a decade — RichardSocher · 2026-10-21
- Startup Electric Plant builds an AI model to decipher plants' electrical signals — TechCrunch AI · 2026-10-21
- Claude reads the literature to build open human surfaceome atlas, claim by claim — owl_posting · 2026-10-20
- Eval researcher teases guide to subjective-task evals, with a curated resource list — cwolferesearch · 2026-10-20
- IIT Madras article splits LLM memory into five approaches and asks why belief revision is still missing — LimeOptimal539 · 2026-10-20
- Scholar writes AI-assisted book on storytelling values, work nominated for EMNLP best paper — _akpiper · 2026-10-20