TailSFT Paper: Filtering Fitted Sequences in SFT Boosts Post-RL pass@1 by up to 4%
rohanpaul_ai · x · 2026-09-04
- arXiv paper "TailSFT: Filtered Fine-Tuning Improves Post-Training Performance" (Sadhika Malladi, Samy Jelassi, et al., Microsoft) argues existing SFT pipelines may not yield checkpoints best suited for RL, building on coverage/pass@K as predictors of post-RL performance.
- Method: filter out already-fit sequences during SFT, focusing learning on the under-modeled tail of the data distribution; design choices validated via controlled experiments and theory.
- Results: on OLMo-3 7B, pass@16 on math/coding improves up to 17% absolute with minimal overhead, and subsequent GRPO runs gain up to 4% absolute pass@1. Includes a lightweight diagnostic for when TailSFT helps, advocating stage-aware checkpoint selection.
Related event: Microsoft's TailSFT: Filtered Fine-Tuning Improves RL Performance(2 posts)→
More from Research
- Dormant poisoning can turn safe LLM agents dangerous after benign fine-tuning; LookAhead defense proposed — mohitban47 · 2026-10-21
- ACS Research Launches Free Platform Giving Researchers Access to Llama 3.1 Base Models — jankulveit · 2026-10-21
- LessWrong Essay Introduces LM Psychology: Just Talking to an AI Still Beats Fancy Evals — jankulveit · 2026-10-21
- LiFT loops a shared DiT core to beat dense baseline by 3.34 FID with 60% fewer params — burny_tech · 2026-10-21
- Stanford researcher clashes over genomics foundation model's cherry-picked variant prediction benchmarks — anshulkundaje · 2026-10-21
- Gemini Agents Crack 59-Year-Old Fusion Math Conjecture in Hours — DynamicWebPaige · 2026-10-21