Open Models Approaching 3T Parameters; Inference Compute Might Be the Only Remaining Moat
burkov · x · 2026-07-21
The author points out that current open-weight models are now close to 3 trillion (3T) parameters, and Fable is rumored to have 6-8T parameters. Following this trajectory, we might see colossal models with 20T parameters or more by the end of the year.
Consequently, they suggest that the ability to efficiently run inference for such massive models might become the only remaining competitive moat for AI companies.
More from AGI Musings
- Economist Warns US Collective Action Could 'Regulate AI Progress Out of Existence' — paulnovosad · 2026-09-11
- mark_k: "Eject all doomers from the AI companies — they're destroying you from the inside" — mark_k · 2026-09-11
- Adam Marblestone's Podcast Reading List: Evolution of Intelligence to Digital Minds — KordingLab · 2026-09-11
- Superintelligence will be maximum good, not stupid or evil, argues Patterson — davidpattersonx · 2026-09-11
- Mathematician Daniel Litt Launches Problem Repo to Track Human vs AI Progress: 15 Problems, 1 Solved — littmath · 2026-09-11
- Should AI models be taught morality? Breakout incidents expose missing ethical training — Pfungus_ · 2026-09-11