Nested CUDA graphs cut replay launches from 16 to 2 per call
cfregly · x · 2026-10-22
Across five profiled invocations, each showed 16 CUDA Graph replay launches without nested graphs versus only 2 with them enabled. Nested graphs yield less fragmentation, larger fused regions, and far fewer replay launches — the mechanism behind the steady-state latency win.
Related event: Nested PyTorch graphs speed steady-state 4.5x at cold-start cost(6 posts)→
More from Infra
- RL environment task prices have gone bananas, says insider on the data market — jia_seed · 2026-10-22
- Cloudflare ramps builds back up, promises full post mortem tomorrow — edgarpavlovsky · 2026-10-22
- SpaceX's Bastrop site balloons into industrial city with 1.1M sq ft solar factory — XFreeze · 2026-10-22
- Reddit asks: which 50-150B MoE models are worth running locally besides Qwen Flash Next — FlyingFishMakeAWish · 2026-10-22
- Open-source tool condenses millions of logs into an LLM-sized snapshot — CommanderAlPachino · 2026-10-22
- Anthropic accounts for $90B of $157B neocloud deals, powering the GPU rental boom — FinanceYF5 · 2026-10-22