Toby Ord: before RLVR, big pre-trains drove most sudden capability jumps
tobyordoxford · x · 2026-11-06
Expanding on his step-change argument, Toby Ord notes that prior to reasoning models and RLVR training, the rarer big new pre-trains were where most new capabilities arrived in sudden jumps. Calculating real annual capability gains requires zooming out to see multiple of these jumps rather than reading a smooth trend.
Related event: Oxford Philosopher Argues AI Capability Gains Rise in Step-Like Jumps(3 posts)→
More from Research
- METR data: Anthropic now pretrains every 45-60 days as capability gains double — zephyr_z9 · 2026-11-06
- Stanford 2.5-hour lecture explains end-to-end how ChatGPT and Claude are built — MaxLi_Columbia · 2026-11-06
- Formal proofs of tens of millions of lines of code are now feasible, says Anthropic engineer — morqon · 2026-11-06
- Hugging Face releases FinePhrase slides: 486B-token synthetic pretraining dataset with 90 rewrite styles — NielsRogge · 2026-11-06
- Toby Ord: jump-plus-steeper-slope mixed hypothesis beats a steeper slope alone — tobyordoxford · 2026-11-06
- SynthFaces: Arc2Face diffusion plus flow matching build balanced synthetic face dataset — Sergey Karpukhin · 2026-11-06