A 37ms Gap Left GPUs Idle ~40% of the Time; Fixing It Removed the Tiny Idles
HankYeomans · x · 2026-10-11
The author shares a hands-on inference performance observation: a seemingly small 37ms gap on the CPU/RAM side, during which the GPU had nothing to do, left the GPUs idle roughly 40% of the time in aggregate. With the gap nearly eliminated, the accumulation of tiny idles is basically gone, markedly improving GPU utilization.
More from Infra
- Dev claims local models can now handle 90% of your work — TheZachMueller · 2026-10-11
- Jevons paradox in AI: falling token prices keep GPU demand tight — AccBalanced · 2026-10-11
- Local AI coding hits only 20-35% of Sonnet's speed in weeks-long app-build tests — julianharris · 2026-10-11
- A tiny logger to find which feature ate the OpenAI budget — and a $113k bill horror story — Pangji1003 · 2026-10-11
- Musk courts chip engineers as Terafab targets 10 chip designs a year — elonmusk · 2026-10-11
- RTX 5090 Hits $5,000 as AI Firms Buy Gaming GPUs by the Pallet — chemist_slime · 2026-10-11