Toby Ord: RL, inference, and pretraining compute show different scaling exponents in preliminary analysis
tobyordoxford · x · 2026-11-14
Oxford philosopher Toby Ord shares preliminary observations on scaling exponent sizes, noting that the slopes of benchmark score vs log(compute) differ between RL compute, inference compute, and pretraining compute — implying distinct power-law exponents for each compute channel. He stresses these are early, informal estimates from limited exploration.
Related event: Oxford's Toby Ord: RL Compute Needs 10,000x Boost to Lift Benchmark Scores(17 posts)→
More from Research
- Cambridge mRNA therapy regrows heart muscle after attack, boosts pumping 10% in mice — kimmonismus · 2026-11-14
- Data curation RL is just SWE envs: code pretraining data increasingly consists of Claude-built, less-sloppy repos — willcb · 2026-11-14
- NeurIPS 2025 best paper: diffusion models start memorizing at ~300 steps per image — Mastbubbles · 2026-11-14
- TestPrune: pruning regression tests to unblock AI-driven autonomous bug fixing — burkov · 2026-11-14
- Toddlers learn new verbs only from responsive partners; a call to rerun the test with AI — AnnaCiaunica · 2026-11-14
- Context beats model choice: Databricks benchmark shows 91% vs 58% for data agents — CautiousUse8597 · 2026-11-14