New notes on a paper argue pretraining determines how far RL can still improve a model
tokenbender · x · 2026-07-21
A detailed readout of “Understanding Reasoning from Pretraining to Post-Training” argues that compute allocation changes with scale.
- Pretraining is the best use of compute up to a certain stage; after that, RL tokens start to matter more.
- SFT and RL affect candidate selection differently: SFT broadens candidate distributions, while RL tends to favor winning among selective candidates.
- Two models with similar size and benchmark scores can have very different RL plasticity depending on how they were pretrained.
The author says this implies we need a measure of a model’s future learnability, not just its current benchmark level. The notes also caution that chess is a limited testbed, and the same intuition is only partly supported when they replicate it on math.
Related event: New Research Proposes Joint Scaling Law for Pretraining and RL(18 posts)→
More from Research
- UI2App shows screenshot fidelity still lags real interaction recovery — Grace Man Chen · 2026-07-22
- Masked diffusion language models boost controllable world models for agentic RL — PatronusAI · 2026-07-22
- Fable 5 reportedly solves one of algebraic geometry’s most famous problems — we_are_mammals · 2026-07-22
- Stanford HAI’s PNAS feature maps the legal questions around generative AI — StanfordHAI · 2026-07-22
- Brain-inspired GCML uses cognitive maps and sampling to plan with less compute — JonLag97 · 2026-07-22
- Proto talk frames generative biology as a high-level programming language — anshulkundaje · 2026-07-22