Open-weight models near 3T parameters as inference emerges as the next moat
ruslansv · x · 2026-07-21
The post argues that open-weight models are already approaching 3T parameters, while a rumored Fable model may have 6–8T parameters. If that trajectory continues, the author expects 20T-scale models by year-end.
The key claim is that inference on such colossal models could become the main moat. The post also notes that training corpora are measured in tens of trillions of tokens, implying that compute-efficient inference may matter more as models and datasets keep scaling.
More from Research
- 3D ResNet Paper Crosses 3,000 Citations Eight Years After CVPR 2018 — HirokatuKataoka · 2026-09-11
- Sample selection and ordering matter a lot in LLM training: DataFlex makes data scheduling dynamic — Puzzleheaded_Box2842 · 2026-09-11
- Jeff Heaton's Intro to the Math of Neural Networks eBook Is Free to Download — blaizedsouza · 2026-09-11
- Mathematician Daniel Litt Launches Problem Repo to Track Human vs AI Progress: 15 Problems, 1 Solved — littmath · 2026-09-11
- Open ECDSA.fail challenge uses AI agents to shrink Shor's-algorithm quantum circuits for Bitcoin keys — StefanoGogioso · 2026-09-11
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11