One cluster alone could train 68 GPT-6-scale models by 2029, and FP4 could double that
scaling01 · x · 2026-09-04
Quoting his own compute-scale tweet, scaling01 calculates that a single large cluster among dozens could support training roughly 68 GPT-6-scale runs by 2029, remarking "we are still so early." He adds he hasn't even factored in FP4 precision, which could make the number 2x-4x higher — an exaggerated but illustrative take on the sheer scale of AI compute buildout and headroom from precision gains.
More from Infra
- Inference startup insider: "we just resell NVIDIA GPUs" — VCs question the moat — firstadopter · 2026-09-04
- Leak claims GPT-6 Astra trained on 100,000+ GPUs at OpenAI's Stargate site — BLUECOW009 · 2026-09-04
- Users dispute credit burn; provider says KV cache was always on, scaling across providers — arthurcolle · 2026-09-04
- Modal adds support for running Cursor Cloud Agents in custom sandboxes — AAAzzam · 2026-09-04
- NVIDIA Jetson initrd flaw lets attackers with physical access bypass Secure Boot — jedisct1 · 2026-09-04
- Ollama's new interactive menu makes launching local models and agents easier — Technovangelist · 2026-09-04