CUDA vet explains why CUDA Graphs beat streams on kernel launch overhead
blelbach · x · 2026-10-21
Answering why enqueuing kernel launches to a stream carries more overhead than a CUDA graph, a veteran who worked on the original CUDA Graphs launch breaks it down: everything reduces to command buffers sent to the GPU. Streams must recreate the buffer (allocation + encoding) on every launch, while graphs compile allocation and encoding ahead of time and reuse the compiled result across launches — so repeated identical work almost always wins with graphs. An old benchmark of linear kernel chains showed both scale linearly, but streams have a zero y-intercept with a steeper slope.
More from Infra
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23
- Public data is drying up; private training data is turning into a browsable catalog — SucceededMind · 2026-10-23