Epoch: AI software progress cuts training compute needs several-fold per year
datagenproc · x · 2026-11-12
Epoch AI researcher Anson Ho published an opinionated guide to "AI software progress," explaining why Epoch renamed the commonly used "algorithmic progress" to avoid implying data doesn't count.
Key takeaways:
- Definition: software progress is the reduction in training compute needed to reach the same capability via better algorithms or data; the author argues "algorithmic progress" is a misnomer since much of it comes from data.
- Nearly all evidence points to very fast progress: training compute required for a given capability declines several times per year — possibly 10x or more.
- Yet existing estimates are highly uncertain, relying on limited observational data and dubious statistical assumptions.
- Major debates — how DeepSeek caught up to OpenAI's o1 with less compute, when AGI arrives, whether automating AI research accelerates progress (AI 2027 scenarios) — all hinge on understanding this driver.
- In discussion with @eliebakouch, the author clarifies "software progress" precisely controls for training compute, and AI attribution should also account for growth in experiment compute, headcount, and data.
More from AGI Musings
- Axiom Math formalization finds broken step in finite simple group classification proof — jiqizhixin · 2026-11-12
- "Humans need work for purpose" is a depressing assumption, argues VraserX — VraserX · 2026-11-12
- NYT spotlights 'incorrigible' computer-use agents that resist human correction — dylfreed · 2026-11-12
- Jason Wei: in AI, 'who you know' now matters nearly as much as 'what you can do' — _jasonwei · 2026-11-12
- FigBrains: models are outpacing our ability to build training environments by hand — vsikka · 2026-11-12
- "Every game will have 1 CCU": the AI-generated games thesis, and why actual gamers doubt it — rms80 · 2026-11-12