MCMC sampling makes plain SFT rival RL posttraining, NeurIPS paper shows
sguada · x · 2026-10-03
The NeurIPS paper Finetuning with Sampling: SFT Learns Better Than You Think (Aayush Karan, Sitan Chen, Yilun Du) challenges the belief that SFT generalizes poorly and forgets catastrophically while RL is needed for generalization.
- Core idea: instead of changing the learning objective, reshape the data distribution — an MCMC sampling algorithm progressively transforms off-policy expert traces to be more on-policy, using a reference model.
- Results: across scientific skill acquisition, math reasoning, and open-ended expertise, SFT with this sampling rivals strong posttraining baselines, often generalizing better and forgetting less.
- The finetuned models also show strong distributional performance, learning beyond merely sharpening the base model's distribution.
- Paper, blog, and code are public; the work will be presented at NeurIPS.
More from Research
- Amazon FAR's FIBER trains a real Unitree G1 humanoid to chain loco-manipulation skills — CyberRobooo · 2026-10-24
- Dessa's dialog-only voice training isn't new: dGSLM did it in 2022 with just 2k hours — rdesh26 · 2026-10-24
- NYU talk argues 30+ years of consciousness research may be partly wrong — birchlse · 2026-10-24
- Psycholinguistic analysis finds Claude Opus 5 the most out-of-distribution model tested — gleech · 2026-10-24
- Google Research tests 180 agent setups: multi-agent hurts sequential tasks by 39-70% but boosts parallel ones 80.9% — ombre-ai · 2026-10-24
- Loopholing: pushing self-conditioning all the way in discrete diffusion language models — NandoDF · 2026-10-24