The cost of instant dispatch in CUDA: stream-path QMDs carry less dependency information
mike64_t · x · 2026-10-22
A thread on CUDA kernel dispatch internals: cuLaunchKernel must dispatch the QMD (queue metadata descriptor) right then and there — the cost of instant dispatch. In the batched-launch path, the resulting QMDs carry less information about true dependencies; the stream path instead spams creation of one-size QMDs with a tracking system for those in flight. The poster finds this design decision odd; another notes batched launching under WSL is painful.
More from Infra
- July 28 MCP release makes remote servers stateless, biggest revision since launch — bibryam · 2026-10-23
- MCP keynote: AAIF hits 305 members, Skills over MCP and Tasks/Events roadmap revealed — bibryam · 2026-10-23
- Only 8% of cloud vendors support OIDC — dev calls to kill long-lived API keys — cramforce · 2026-10-23
- Swift-1.5-Qwen3.8-27B ported to hipfire MQ4: 2.2x faster end-to-end than GGUF on a 7900 XT — Queasy_Asparagus69 · 2026-10-23
- Intel CTO: chip output can at best 2X while AI compute grows 3X per year — SammaVaco · 2026-10-23
- Engineer reverse-engineers projector TV's 10s wake lag to a cold-booting Linux board — banteg · 2026-10-23