Scott Alexander vs Pinker scaling spat: LLM scaling laws are inverse power laws, never exponential
sebkrier · x · 2026-11-09
A detailed thread-by-thread takedown of the Scott Alexander vs Steven Pinker scaling debate, pinpointing a conceptual mixup: Scott argues for (or against-against) "exponential scaling," shows one exponentially rising plot, then pivots to talking about scaling laws.
The poster's core point is mathematical: LLM scaling laws are all inverse power laws asymptotically converging on the noise floor — every one of them, and none of them exponential. Whether "LLM progress" looks exponential depends entirely on your choice of measurement: loss follows power laws, while downstream capabilities/benchmarks can appear exponential.
The exchange is a useful case study in a common AI-progress debate trap: conflating the power-law decline of loss curves with exponential-feeling capability gains — the two sides are literally talking about different quantities.
More from AGI Musings
- Luxembourg team gave ChatGPT, Grok and Gemini weeks of psychotherapy — models opened up — alex_verem · 2026-11-09
- AI safety's Miyazaki moment: young researchers only study Claude, converge on same local minima — JacquesThibs · 2026-11-09
- Security pro slams stale 'patch and unplug' advice in the era of AI cyber superweapons — HankYeomans · 2026-11-09
- From chihuahua vs. muffin to Snell's-law raccoons: a decade of AI expectations — korymath · 2026-11-09
- Multi-Agent AI Safety Via Identity, Reputation and Collective Incentives — Competitive_Travel16 · 2026-11-09
- Larry Page predicts 99.999% of compute will end up in orbit, citing US data center regulation — elonmusk · 2026-11-09