Open Models Approaching 3T Parameters; Inference Compute Might Be the Only Remaining Moat
burkov · x · 2026-07-21
The author points out that current open-weight models are now close to 3 trillion (3T) parameters, and Fable is rumored to have 6-8T parameters. Following this trajectory, we might see colossal models with 20T parameters or more by the end of the year.
Consequently, they suggest that the ability to efficiently run inference for such massive models might become the only remaining competitive moat for AI companies.
More from AGI Musings
- Claude Code skill uses 10 Markdown rules to make outputs ADHD-friendly — alex_verem · 2026-07-22
- AI Power Demand Exposes US Energy Gap, Urging Shift from Scarcity to Abundance — bradneuberg · 2026-07-22
- ControlAI CEO says an international ban on superintelligence is needed to avert extinction risk — zetalyrae · 2026-07-22
- Gary Marcus says LLMs still cannot really do math on their own — GaryMarcus · 2026-07-22
- Gary Marcus says LLM math skills are like knowing only a car’s engine size — GaryMarcus · 2026-07-22
- AI may make digital work infinitely leveraged while offline life gets more human — illscience · 2026-07-22