Training 1 FLOP, inference 6 FLOP: a $220B inference economy
ravikantagrawal · x · 2026-11-07
A discussion on whether inference matters more than training: roughly six FLOPs go to serving inference each year for every FLOP spent training, and getting inference right has produced over $220 billion in market cap in the past decade. Inference involves a complex stack of routing, caching, scheduling, kernels, memory, and hardware balancing speed, cost, and quality.
More from Infra
- NVIDIA's first co-packaged optics Ethernet switch, Spectrum-X Photonics, enters full production — TinfoilTricorn · 2026-11-07
- This week's open training stack: MoE training handbook, harness adapters, and a harness leaderboard — ariG23498 · 2026-11-07
- Musk: space will be the cheapest place to run superintelligence within 2-3 years — TinfoilTricorn · 2026-11-07
- JITEngine synthesizes custom LLM inference engines for Apple Silicon, claims up to 2.7x speedup — AccBalanced · 2026-11-07
- Dev predicts 2027's local inference boom: integrated hardware + OS + inference stack, not just GPUs — vincent_koc · 2026-11-07
- vLLM Office Hours Hits 59 Sessions: 76 Speakers from 24 Organizations and Counting — vllm_project · 2026-11-07