GPU vs CPU economics: MFU above 30% is common, but is it FLOPs well spent?
joseph_h_garvin · x · 2026-10-30
A technical debate on GPU utilization and compute economics.
josephhgarvin argues that in CPU-world, where work is small and software complicated, almost nothing runs close to optimal; in GPU-world, where work is expensive and software simple, things run quite close to optimal, with MFU above 30% being common.
A critic counters that MFU can miss the forest for the trees: asking countless LLM instances to each execute inherently sequential, branchy algorithms (without code execution) would look great on MFU while most FLOPs go unproductively used—a CPU run would be much cheaper. cisfemale responds that LLM instances couldn't run such algorithms without code execution anyway, and that GPUs handle sequential/branchy code better than expected, though still worse than CPUs.