Model Scale and Data Emerge as New Bottlenecks
willccbb · x · 2026-07-12
This reply discusses several key constraints on the future development of AI:
- Demand side: Near-infinite token demand will continue to push model scales larger.
- Supply side: Limited chip availability means hardware will become a bottleneck for expansion.
- Training environments: Once the cost of environment construction approaches zero, the advantage will shift more towards raw data.
The author also believes that the truly worthwhile approach is to look ahead along this trend: there will be many future opportunities, and openly sharing "secret tricks" will actually benefit hiring, fundraising, and marketing, as everyone is eager to understand these shifts.
Related event: New AI Labs Use Technical Reports to Break Out(3 posts)→
More from AGI Musings
- AI era exposes how often human pride beats basic reasoning — haider1 · 2026-07-21
- FactoryAI’s Enoreyes says model distillation is basically unstoppable — LangChain · 2026-07-21
- Andrew Blumberg says formalization without interpretability is not science — AlexKontorovich · 2026-07-21
- Ken Ono says AI is forcing mathematicians to rethink how discovery works — soumitrashukla9 · 2026-07-21
- Open-source labs could distill a state-of-the-art model to 32GB or 80GB VRAM, the post argues — bookwormengr · 2026-07-21
- Two US companies are now using superintelligence to speed up the next generation of models — yacineMTB · 2026-07-21