Back-of-Envelope: Frontier US Model Estimated at 8e26 FLOPs, 800B Active Params, 170T Training Tokens
teortaxesTex · x · 2026-09-15
TeortaxesTex extrapolates from Liang Wenfeng's May figure about the largest model known to be in development by Americans: assuming fp4 precision on Blackwells, 8.17e26 FLOPs, 800B active params, and a reasonable 4%-5% sparsity (16-20T total params), such a model would need roughly 170T training tokens. He adds that OpenAI can already access 100T+ tokens (citing glm-oss-related data). Speculative but source-anchored estimate of frontier model scale and training resources.
More from Infra
- To dethrone Postgres, new databases must prioritize ecosystem integrations from day one — mgill25 · 2026-11-02
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02