Microsoft TailSFT boosts code pass@16 by up to 16.8 points
dair_ai · x · 2026-08-31
Microsoft researchers propose TailSFT, a method to improve the standard SFT pipeline for better Reinforcement Learning performance.
- Problem: Standard SFT continues spending gradient on sequences the model has already fit, narrowing the distribution RL needs to explore later.
- Method: TailSFT filters out these sequences during training, focusing learning on the under-modeled tail of the data.
- Results: On OLMo-3 7B, pass@16 improves by up to 16.8 points absolute on coding and 3.1 on math. These higher-coverage checkpoints lift final pass@1 after GRPO by up to 3.9 points, and in some settings, early reward climbs 2.5x faster than matched standard SFT runs.
More from Research
- A beginner-friendly introduction to causal inference kicks off, Part 1 — Nasereliver · 2026-10-16
- FBI method trains for Pass@K with one sample per update at 12.3% of the cost — EmtiyazKhan · 2026-10-16
- Berkeley researchers sample humanoid robot policies directly from weight space with diffusion — berkeley_ai · 2026-10-16
- OneWM: unified causal training shows world and action modeling improve each other — IAAI233 · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16
- 5.8M-param Speech Event Detector adds laughs, breaths and背景 sounds to Whisper transcripts — Ayush Kumar Bar · 2026-10-16