Qwen's Junyang Lin: Light SFT + Benchmarks Can't Reveal a Base Model's True Potential
_AndrewZhao · x · 2026-11-19
Asked whether there's any way to evaluate a base model other than an actual post-training phase, Junyang Lin (JustinLin610) replied that light SFT plus benchmarks is an old method with a fundamental flaw: teams focus too much on post-train benchmark scores instead of judging the model's underlying potential — whether it is knowledgeable and smart at reasoning. He added that he doesn't think there's a deterministic way to solve this problem. A candid take from a core Qwen team member on why benchmark scores systematically misalign with base-model potential.
Related event: Qwen's Lin: Lightweight SFT Benchmarks Can't Reveal Base Model Potential(2 posts)→
More from Models
- Google Ships Gemini 4 Neon, Positioned Between Carbon and Hydrogen for Fast Agentic Work — jeremyphoward · 2026-11-22
- Microsoft's USB4 RDMA lets mini PCs pool memory to run Qwen3-235B — JFPuget · 2026-11-22
- Google AI Mode now enforces usage limits, users surprised by rate-limit popup — elvano124 · 2026-11-22
- Sol 6.2 now refuses reverse engineering that Sol 6.1 happily did for a month — cardak98 · 2026-11-22
- Anthropic's record ~$81B IPO, GPT-6.2 pricing, Qwen4-Max and Copilot Autopilot: week in review — ksraj1001 · 2026-11-22
- Reddit rant: Frontier AI labs are training models to imitate lazy engineers — RossPeili · 2026-11-22