Ambient SFT: temperature matching fixes catastrophic forgetting in fine-tuning, gains +20 IFEval without losing old skills

giannis_daras · x · 2026-11-19

Researcher Giannis Daras introduces Ambient SFT, a new fine-tuning method targeting SFT's biggest flaw: learning a new task at the cost of forgetting old capabilities.

The problem: vanilla SFT forces a bad trade-off — Qwen3-1.7B fine-tuned on 20K instruction examples gains 21 points on IFEval but loses up to 10 points on knowledge, math and coding benchmarks. Early stopping and other regularizers only pick a point on the trade-off, not remove it.

The method (temperature matching): treat the pre-trained model as a noisy teacher and add a KL regularizer matching the model's predictions at a coarse, high-temperature level (loss = SFT loss + λ·KL). Temperature T is the single knob: T=1 is classic knowledge distillation, T→∞ is vanilla SFT, and the optimum sits in the middle (T=2.5 for Python code domain adaptation). The idea connects to Ambient Diffusion — an LLM trained only on high-temperature gibberish can still learn to write English.

Results (1.7B–14B, Qwen/OLMo/Llama, across domain adaptation, instruction tuning and reasoning):

Related event: Ambient SFT: Plug-and-Play Fine-Tuning That Avoids Forgetting(4 posts)→

Original post →

More from Research

Research channel →