Study finds warp divergence still scales linearly across Pascal, Ampere, Hopper and Blackwell
AlpinDale · x · 2026-07-28
Cross-generation study finds warp divergence still serializes linearly on NVIDIA GPUs
A short arXiv paper characterizes warp divergence from Pascal through Ampere, Hopper, and Blackwell using cycle-accurate microbenchmarks, hardware counters, and static SASS analysis.
Key findings:
- Divergent paths serialize roughly linearly with the number of paths, with no super-linear reconvergence penalty.
- Warp execution efficiency drops as 32/k, and the penalty appears independent of occupancy.
- Predication removes the serialization cost.
- The same programmer-visible cost model already holds on pre-ITS Pascal, so the behavior predates Volta’s Independent Thread Scheduling.
- What changed is the compiler-emitted reconvergence machinery: Pascal uses SSY/SYNC stacks, while newer generations use barrier-register instructions.
- Blackwell adds two-tier convergence-barrier classification, uniform-branch instructions, and explicit partial-mask warp synchronization.
The authors conclude that divergence remains a stable and predictable performance cost, even as the control-flow ISA and reconvergence mechanisms keep evolving.
Related event: NVIDIA GPU Warp Divergence Cost Remains Linear Across Generations(2 posts)→
More from Infra
- US power grid proposes making data centers pay fully for the electricity they use — pstAsiatech · 2026-07-28
- A blunt rebuttal to the idea that data-center debt is the new subprime crisis — JOBhakdi · 2026-07-28
- Nvidia’s Texas data-center lease is $19.6B for 15 years, not $50B upfront — firstadopter · 2026-07-28
- Analyst says Google will need Intel supply by 2028, not just TSMC — zephyr_z9 · 2026-07-28
- Search API pricing looks cheap per call, but full-context queries can cost 4× more — No_Marionberry_5366 · 2026-07-28
- Pentagon reportedly moves to build hyperscale AI data centers on a dozen bases — Polymarket · 2026-07-28