Qwen's Junyang Lin: Light SFT + Benchmarks Can't Reveal a Base Model's True Potential

_AndrewZhao · x · 2026-11-19

Asked whether there's any way to evaluate a base model other than an actual post-training phase, Junyang Lin (JustinLin610) replied that light SFT plus benchmarks is an old method with a fundamental flaw: teams focus too much on post-train benchmark scores instead of judging the model's underlying potential — whether it is knowledgeable and smart at reasoning. He added that he doesn't think there's a deterministic way to solve this problem. A candid take from a core Qwen team member on why benchmark scores systematically misalign with base-model potential.

Related event: Qwen's Lin: Lightweight SFT Benchmarks Can't Reveal Base Model Potential(2 posts)→

Original post →

More from Models

Models channel →