Back-of-envelope: 800k H100s could serve a ~19.2T-parameter FP8 model's weights
Gokbagbasi · reddit · 2026-10-17
A Reddit thought experiment estimates the parameter ceiling for a hypothetical GPT-6-class model: allocating 30% of 800k H100-equivalents (240k GPUs, 19.2 PB memory at 80 GB each) gives roughly 19.2T parameters in FP8 at 1 byte/param, before KV cache, runtime overhead, and redundancy. Serving 10k–100k concurrent users at 66 tokens/s means 660k–6.6M output tokens/s, which depends on architecture, batching, bandwidth, and parallelism rather than peak FLOPS. The author stresses all figures are hypothetical, not verified claims about OpenAI's infrastructure, and invites feedback from large-scale inference practitioners.
More from Infra
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23
- Public data is drying up; private training data is turning into a browsable catalog — SucceededMind · 2026-10-23