Joint pretraining-RL scaling says RL share should rise with compute
teortaxesTex · x · 2026-07-20
A scaling-law discussion on joint pretraining and RL suggests that the optimal RL share rises with total compute, but the pretraining allocation itself does not deviate much from Chinchilla-style scaling.
The author highlights a synthetic chess setting where reward, pretraining loss, and RL compute are linked, and argues that for pass@1, pretraining affects results in two ways: it lowers downstream loss and improves how efficiently RL compute turns into reward. The post also warns that naively extrapolating the fitted law to very large compute can produce impossible rewards and wildly inflated RL shares, so the shared scaling trend should be kept, but capped conservatively.
Related event: New Research Proposes Joint Scaling Law for Pretraining and RL(18 posts)→
More from Research
- MIT's Ziming Liu: AutoResearch doesn't generalize — blind training tricks cause 'forking' disasters — ZimingLiu11 · 2026-10-26
- Stanford Data-Centric AI Workshop reopens site, extends poster deadline — jure · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Ito: a browser that detects browsing intent and auto-organizes tabs into Flows — andrewkuznet · 2026-10-26
- Cornell study says brains process language as linear sequences, challenging Chomsky's hierarchy — anirbanbandyo · 2026-10-26
- Hi3D V1.0 open-sourced with 2M training assets and 90% slicing success rate — kimmonismus · 2026-10-26