Toby Ord: RL post-training compute now rivals pretraining, further 10x scaling gets hard
tobyordoxford · x · 2026-10-14
- Toby Ord cites a chart showing RL post-training compute reached 100% of pretraining size — the very quarter he predicted in his Oct 2025 essay; he had earlier suggested RL post-training exceeded 10% of pretraining compute, and a dubious xAI graph implied parity.
- Once at 100%, a further 10x means 10x-ing the total training budget — and labs would rather spend on far more compute-efficient pretraining.
- But pretraining scaling may be blocked by a lack of good training data after ingesting all human books. The chart suggests RL… (truncated)
Related event: Toby Ord: RL Post-Training Compute Now Matches Pretraining(2 posts)→
More from AGI Musings
- alphaXiv Maps 10,000 Papers: RL Post-Training Fades as Agents, Memory, Robotics Heat Up — JayAlammar · 2026-10-14
- Artificial neurons don't physically exist — an LLM is just matrix math — ValerioCapraro · 2026-10-14
- The moats debate: classics survive, but agents are driving switching costs to zero — lennysan · 2026-10-14
- Reddit user: the only limits left in the AI era are creativity and tokens — masoomdon · 2026-10-14
- Heavy ChatGPT user since 2022 asks: has AI been a net positive for your life? — Alternative-Car8221 · 2026-10-14
- The AI Layoff Trap: Rational Firms Automate Into a Collective Loss, Paper Finds — mdancho84 · 2026-10-14