NVIDIA warp divergence still scales linearly from Pascal to Blackwell, paper finds
Alpin Dale · hf · 2026-07-28
What the paper finds
This study re-examines warp divergence on NVIDIA GPUs from Pascal to Blackwell using cycle-accurate microbenchmarks, hardware counters, and compiler-generated SASS analysis.
Key results
- Divergent paths serialize linearly with the number of paths k: T(k) ≈ s·k
- No super-linear reconvergence penalty was observed
- Warp execution efficiency drops as 32/k
- The penalty is independent of occupancy
- Predication removes the serialization cost
- The same programmer-visible cost model already exists on Pascal, so it predates ITS
ISA/compiler changes
- Pascal uses a per-warp SSY/SYNC instruction stack
- Newer generations use barrier-register instructions instead
- Deferred reconvergence beyond the immediate post-dominator drops from 29 cases on Ampere to 2 on Blackwell
- Blackwell adds a two-tier convergence-barrier classification, uniform-branch instructions, and explicit partial-mask warp synchronization
- Bit-flip experiments suggest the new barrier class is a static compiler classification with no observable runtime effect in these tests
Bottom line
Warp divergence keeps a stable, predictable performance cost across generations, even as NVIDIA’s control-flow ISA and reconvergence machinery continue to evolve.
Related event: NVIDIA GPU Warp Divergence Cost Remains Linear Across Generations(2 posts)→
More from Infra
- Neocloud stocks sink on Gartner's "80% will fail" report; analyst calls it garbage — McDonaghMatthew · 2026-10-16
- JapanFold Claims One Tenstorrent Server Beats an Nvidia DGX B200 on Bio Models at a Third of the Price — DavidBennett__ · 2026-10-16
- vLLM ships Day-0 RL support for Beam in SkyRL, +62% LoRA decode throughput — vllm_project · 2026-10-16
- DFlash2 speeds Beam decode up to 3.03× across benchmarks, vLLM reports — vllm_project · 2026-10-16
- Inside Beam: 501B params, only 23B active per token via 8-of-256 expert routing — vllm_project · 2026-10-16
- CoreWeave founders cashed out $3.3 billion before the stock's meltdown — SumitGup · 2026-10-16