The scaling debate hinges on two readings of "predictably better with scale"
Diyi_Yang · x · 2026-10-07
In a discussion involving Percy Liang, Diyi Yang and others, William Barr Held points out that much of the scaling controversy comes from conflating two readings of the core idea:
- Models get predictably better with scale — a big model will almost always beat a small one.
- Models get predictably better with scale — you can make meaningful statements about large models based on small ones.
Which interpretation people latch onto explains why the debate keeps talking past itself.
More from AGI Musings
- Gary Marcus: Vague AI Math Proof Report Would Never Pass Peer Review — GaryMarcus · 2026-10-08
- Rotman Professor Sums Up His AI and Management Course in One Post — Afinetheorem · 2026-10-08
- eigenrobot on Academic Fraud: Cover-ups and Threats Are the Norm, Not the Exception — eigenrobot · 2026-10-08
- Researcher estimates 25% chance some current AIs are already conscious — cccalum · 2026-10-08
- A debate on whether AI abundance is distributed equally to everyone — LeviTurk · 2026-10-07
- AI Ads for AI Apps for an Audience Being Replaced by AI: 'The Circularity Is Here' — sull · 2026-10-07