Ambient SFT: temperature matching fixes catastrophic forgetting in fine-tuning, gains +20 IFEval without losing old skills
giannis_daras · x · 2026-11-19
Researcher Giannis Daras introduces Ambient SFT, a new fine-tuning method targeting SFT's biggest flaw: learning a new task at the cost of forgetting old capabilities.
The problem: vanilla SFT forces a bad trade-off — Qwen3-1.7B fine-tuned on 20K instruction examples gains 21 points on IFEval but loses up to 10 points on knowledge, math and coding benchmarks. Early stopping and other regularizers only pick a point on the trade-off, not remove it.
The method (temperature matching): treat the pre-trained model as a noisy teacher and add a KL regularizer matching the model's predictions at a coarse, high-temperature level (loss = SFT loss + λ·KL). Temperature T is the single knob: T=1 is classic knowledge distillation, T→∞ is vanilla SFT, and the optimum sits in the middle (T=2.5 for Python code domain adaptation). The idea connects to Ambient Diffusion — an LLM trained only on high-temperature gibberish can still learn to write English.
Results (1.7B–14B, Qwen/OLMo/Llama, across domain adaptation, instruction tuning and reasoning):
- Learns the task just as well (IFEval +20) while mostly keeping prior knowledge
- Even gains new capabilities: MATH +7, GSM8K +20
- Up to -26.7% perplexity vs best baseline on domain adaptation, +8–20% average gain over SFT on instruction tuning, +3.5% relative gain on reasoning
Related event: Ambient SFT: Plug-and-Play Fine-Tuning That Avoids Forgetting(4 posts)→
More from Research
- Fine-tuned Dream-1 world action model jumps 159% on real-robot benchmark at 1/100th the cost — paigeinsf · 2026-11-19
- AI makes 'living papers' nearly free to build, but their durability is doubtful — _akpiper · 2026-11-19
- HiT image generator hits 1.48 FID learning hierarchy from pixels, fast reproduced — niloofar_mire · 2026-11-19
- DaisyRose-27B: RL on 144 H200s for 18 days lifts Terminal-Bench from 5.6% to 12.0% — ying11231 · 2026-11-19
- After a fix, MuonM edges out Muon while MuonS performs about the same — YouJiacheng · 2026-11-19
- GPT 5.6 Sol, released just 4 months ago, can no longer reliably monitor frontier models — connoraxiotes · 2026-11-19