Distillation debate: SFT era is over, RL environments drive capability gains now
xeophon · x · 2026-09-25
A debate on X about the role of distillation: xeophon argues the claim that distillation is why Chinese models approach US ones is outdated — SFT as the main source of capability gains was a 2023-24 story, and by 2026 progress comes from RL environments. He adds that distillation also serves as a way to offload compute, especially when leveraging questionable subscription accounts for data.
The context was a question about why labs still invest so heavily in distillation: it's framed more as a compute-cost strategy than a driver of frontier capability.
Related event: AI Researchers Debate Whether Distillation Drives China's Model Gains(5 posts)→
More from AGI Musings
- Nat Friedman's personal creed: "Slow is fake," a week is 2% of the year, and the EMH is a lie — alexeyguzey · 2026-09-25
- P(Doom) conflates two questions: is AI dangerous, and can humans respond? — binarybits · 2026-09-25
- Why P(Doom) is a flawed construct: it depends on society's reaction function — binarybits · 2026-09-25
- Prophetic sects before societal shifts: what AI institution-builders can learn from history — lawhsw · 2026-09-25
- Founder argues datacenter buildouts are a dead end — AI models will shrink like mainframes did — draginol · 2026-09-25
- AGI safety debate: is 'just don't build AGI' the option being falsely excluded? — JacquesThibs · 2026-09-25