Distillation debate: RL, not distilling from sol, likely explains the model's gains

JoshPurtell · x · 2026-09-03

JoshPurtell pushes back on claims that a certain model distilled from sol:

Synthesizing training data with sol offers almost no benefit over Kimi k3 or GLM 5.3 — the latter are actually cheaper
Key evidence: the model outperforms sol after training, so RL is the big part of the story
If they distilled, they wouldn't add an SFT stage to the RL pipeline (SFT+RL is harder than RL-only), so distillation is possible but "not 80%"

For long-horizon agent developers distillation is tempting; for trivial one-shot structured-output tasks with gold outputs, synthesizing data is trivially easy.

Original post →

Distillation debate: RL, not distilling from sol, likely explains the model's gains

More from Models