New notes on a paper argue pretraining determines how far RL can still improve a model
tokenbender · x · 2026-07-21
A detailed readout of “Understanding Reasoning from Pretraining to Post-Training” argues that compute allocation changes with scale.
- Pretraining is the best use of compute up to a certain stage; after that, RL tokens start to matter more.
- SFT and RL affect candidate selection differently: SFT broadens candidate distributions, while RL tends to favor winning among selective candidates.
- Two models with similar size and benchmark scores can have very different RL plasticity depending on how they were pretrained.
The author says this implies we need a measure of a model’s future learnability, not just its current benchmark level. The notes also caution that chess is a limited testbed, and the same intuition is only partly supported when they replicate it on math.
Related event: New Research Proposes Joint Scaling Law for Pretraining and RL(18 posts)→
More from Research
- Mathematician Daniel Litt Launches Problem Repo to Track Human vs AI Progress: 15 Problems, 1 Solved — littmath · 2026-09-11
- Open ECDSA.fail challenge uses AI agents to shrink Shor's-algorithm quantum circuits for Bitcoin keys — StefanoGogioso · 2026-09-11
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11
- Navier-Stokes, Riemann, P vs NP: what this week's math buzzwords mean for you — koltregaskes · 2026-09-11
- Fruit fly brain as an LLM: connectome-driven language model demo goes live — ngxson · 2026-09-11
- Harry Collins: LLMs can't do frontier science because they can't invent new language — whoamisri · 2026-09-11