MIT paper: scaling law expiring as cost doubles 6x per width jump
DavidLinthicum · x · 2026-09-26
David Linthicum summarizes a new MIT paper challenging AI's "bigger is better" era. Doubling model width halves error but multiplies training cost 6x: from today's $300M frontier models to $1.8B in one doubling and $11B in two, with no viable economic path. Even with free money, the error floor is locked by Zipf's law — the natural frequency distribution of language itself won't move, putting hard limits on scaling gains.
More from Infra
- SpaceX's Memphis supercomputer: millions of GPUs and over two gigawatts of compute — CurieuxExplorer · 2026-09-26
- The handiest GPU this dev ever bought is a ~$300 Intel Arc A310, not NVIDIA or AMD — TheZachMueller · 2026-09-26
- Running image generation in the browser on local hardware: 10s pixel art on an RTX 3060 — Bartholomheow · 2026-09-26
- Alibaba's T-Head unveils Zhenwu V900 chip: 216GB per card, sales in Q1 2027 — shashib · 2026-09-26
- llama.cpp fork dedups repeated prompts losslessly, cutting 108k to 71k tokens in agent loops — Odd_Cauliflower_8004 · 2026-09-26
- Why rent servers when agents can run your terminal? — StewartalsopIII · 2026-09-26