New notes on a paper argue pretraining determines how far RL can still improve a model
tokenbender · x · 2026-07-21
A detailed readout of “Understanding Reasoning from Pretraining to Post-Training” argues that compute allocation changes with scale.
- Pretraining is the best use of compute up to a certain stage; after that, RL tokens start to matter more.
- SFT and RL affect candidate selection differently: SFT broadens candidate distributions, while RL tends to favor winning among selective candidates.
- Two models with similar size and benchmark scores can have very different RL plasticity depending on how they were pretrained.
The author says this implies we need a measure of a model’s future learnability, not just its current benchmark level. The notes also caution that chess is a limited testbed, and the same intuition is only partly supported when they replicate it on math.
Related event: New Research Proposes Joint Scaling Law for Pretraining and RL(18 posts)→
More from Research
- MIT's Ziming Liu: AutoResearch doesn't generalize — blind training tricks cause 'forking' disasters — ZimingLiu11 · 2026-10-26
- Stanford Data-Centric AI Workshop reopens site, extends poster deadline — jure · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Ito: a browser that detects browsing intent and auto-organizes tabs into Flows — andrewkuznet · 2026-10-26
- Cornell study says brains process language as linear sequences, challenging Chomsky's hierarchy — anirbanbandyo · 2026-10-26
- Hi3D V1.0 open-sourced with 2M training assets and 90% slicing success rate — kimmonismus · 2026-10-26