TailSFT: skip already-learned SFT examples to boost post-RL pass@k
canondetortugas · x · 2026-09-12
- DeepMind researchers propose TailSFT, a lightweight SFT tweak that improves coverage and post-RL performance.
- Motivation: RL is expensive; prior work (1yr ago) showed vanilla cross-entropy SFT isn't the best RL prep.
- Method: record each example's pre-SFT loss L0; during SFT, exclude the examples most-improved-relative-to-L0 from backprop (sort-and-slice or zero-masking), since the model has already learned them.
- Result: better pass@k starting point for k>1, better post-RL results. Lucas Beyer calls it simple and intuitive.
Related event: DeepMind and Princeton Propose TailSFT to Boost RL Performance(3 posts)→
More from Models
- Open TTS Leaderboard adds 14 languages, new LFM3-Audio-2B and maya2-native models — realmrfakename · 2026-10-31
- Developers say the Linux Codex app is broken and neglected — eigenron · 2026-10-31
- Kimi now accepts Russian Mir cards directly: Pro subscription at $39/month without intermediaries — teortaxesTex · 2026-10-31
- Nat Lambert: Open models could be what holds the AI ecosystem together — natolambert · 2026-10-31
- Gemini 4 Carbon debuts #1 on APEX-Agents, the only model averaging above 90% — Swarooprm7 · 2026-10-31
- Sarvam quietly builds an India-focused opencode with 'local inference', gets mocked for the definition — kingslyj · 2026-10-31