OLMo results show pretraining loss still predicts RL gains on math reasoning
Pavel_Izmailov · x · 2026-07-21
The paper extends the chess scaling results to natural-language math reasoning with OLMo models.
Main points:
- Pretraining loss still strongly predicts later RL performance.
- The relationship becomes steeper as pretraining tokens increase.
- RL does not act uniformly: it tends to sharpen policies on easy tasks, but on harder tasks it can sometimes uncover tail solutions that were very unlikely under the original policy.
Taken together, the results suggest that pretraining quality and quantity shape not just baseline capability, but also how effectively RL can improve a model later on.
Related event: New Research Proposes Joint Scaling Law for Pretraining and RL(18 posts)→
More from Research
- MIT's Ziming Liu: AutoResearch doesn't generalize — blind training tricks cause 'forking' disasters — ZimingLiu11 · 2026-10-26
- Stanford Data-Centric AI Workshop reopens site, extends poster deadline — jure · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Ito: a browser that detects browsing intent and auto-organizes tabs into Flows — andrewkuznet · 2026-10-26
- Cornell study says brains process language as linear sequences, challenging Chomsky's hierarchy — anirbanbandyo · 2026-10-26
- Hi3D V1.0 open-sourced with 2M training assets and 90% slicing success rate — kimmonismus · 2026-10-26