Nat Lambert Clarifies LLM Distillation Mechanics
Nat Lambert refuted the misconception that Chinese labs use the strongest models directly as teachers during RL, emphasizing that distillation actually plays a crucial role primarily during the SFT and midtraining phases.
2026-07-21 ~ 2026-07-22 · 2 related posts
- Nat Lambert says RL distillation does not use the strongest models as teachers — natolambert · 2026-07-21
- Nat Lambert argues distillation mostly matters in SFT and midtraining, not RL — natolambert · 2026-07-22