Nested CUDA graphs trade 33.2% longer cold start for faster steady state
cfregly · x · 2026-10-22
Follow-up measurement: with nestedgraphbreaks enabled, the sum of the first ten synchronized compiled-call intervals rose from 2.411s to 3.212s (+33.2%), with nearly all the overhead concentrated in call 1 (0.79–0.88s extra). Calls 2–10 were faster. Net verdict: a steady-state win with a disclosed cold-start cost — measure first use separately from repeated execution.
More from Infra
- Hugging Face ships official Kernels tutorial: drop prebuilt Hub kernels into your PyTorch model — ariG23498 · 2026-10-22
- HUMAIN and NVIDIA plan up to 84MW of AI compute in Saudi Arabia, ~27,000 GPUs — HUMAIN · 2026-10-22
- Jensen Huang: Energy is AI's biggest bottleneck, time to invest in all forms of energy production — kimmonismus · 2026-10-22
- NVIDIA unveils Vera Rubin full-stack agent hardware, claims post-trained Nemotron beats frontier models — mervenoyann · 2026-10-22
- Meme: all of modern digital infrastructure rests on one ASML machine — VoidAsuka · 2026-10-22
- OrbStack adds self-hosting for orbs, free for individuals and teams — jasonkneen · 2026-10-22