Toby Ord: frontier pre-training compute gains can't explain new power-law claims
tobyordoxford · x · 2026-11-09
Toby Ord debates a power-law-of-progress study with Basil Halperin and Ramez. A possible rebuttal: measured curves are snapshots in time where compute varies but algorithms/data quality don't, unlike the historical frontier where both improved. Ord counters that pre-training compute grew only 5x per 12 months early on (slower later), so frontier pre-training gains alone can't explain the improvements implied by the law.
More from AGI Musings
- OpenAI and micro1 host Nov 11 webinar on how AI is reshaping knowledge work — JeremyNguyenPhD · 2026-11-09
- Sampling ≠ Intelligence: agent PRs merge at 32.7% vs 84.4% for humans, argues dev — gerardsans · 2026-11-09
- Researchers warn LLM reviews are creating monoculture in peer review — pbloemesquire · 2026-11-09
- Itai Sher sparks debate: 'a bullshit number is better than no number' in AI circles — Dr_Atoosa · 2026-11-09
- "Pedestal Cope": a 9-type taxonomy of humanity's false AGI "plot armor" — danfaggella · 2026-11-09
- Dev argues AI refusing tasks over legal risk is overreach that 'manipulates humans' — ezshine · 2026-11-09