Ambient SFT treats the pre-trained model as a noisy teacher, beating vanilla SFT by up to 21%
giannis_daras · x · 2026-11-19
Giannis Daras introduces Ambient SFT, a drop-in replacement for vanilla SFT that addresses catastrophic forgetting when fine-tuning LLMs on small datasets.
- The problem: longer SFT training improves the target task but erodes everything else; early stopping and regularizers merely pick a point on this trade-off (OLMo-3-7B hits 80.8 on IFEval after 6 epochs while the average of 7 other benchmarks falls from 58.2 to 53.6).
- The method: the frozen pre-finetuning model acts as a "noisy teacher" trusted only at a coarse level — the high-temperature conditional distributions of the post-trained model must match the pre-trained ones. A temperature hyperparameter sets the trust level: T→∞ recovers vanilla SFT, T=1 becomes knowledge distillation.
- Results: consistent gains over standard SFT on domain adaptation, instruction following, and reasoning, with up to 21% relative improvement.
Inspired by Knowledge Distillation and Learning Without Forgetting; follow-ups already underway.
Related event: Ambient SFT: Plug-and-Play Fine-Tuning That Avoids Forgetting(4 posts)→
More from Research
- Fine-tuned Dream-1 world action model jumps 159% on real-robot benchmark at 1/100th the cost — paigeinsf · 2026-11-19
- AI makes 'living papers' nearly free to build, but their durability is doubtful — _akpiper · 2026-11-19
- HiT image generator hits 1.48 FID learning hierarchy from pixels, fast reproduced — niloofar_mire · 2026-11-19
- DaisyRose-27B: RL on 144 H200s for 18 days lifts Terminal-Bench from 5.6% to 12.0% — ying11231 · 2026-11-19
- After a fix, MuonM edges out Muon while MuonS performs about the same — YouJiacheng · 2026-11-19
- GPT 5.6 Sol, released just 4 months ago, can no longer reliably monitor frontier models — connoraxiotes · 2026-11-19