Study finds warp divergence still scales linearly across Pascal, Ampere, Hopper and Blackwell
AlpinDale · x · 2026-07-28
Cross-generation study finds warp divergence still serializes linearly on NVIDIA GPUs
A short arXiv paper characterizes warp divergence from Pascal through Ampere, Hopper, and Blackwell using cycle-accurate microbenchmarks, hardware counters, and static SASS analysis.
Key findings:
- Divergent paths serialize roughly linearly with the number of paths, with no super-linear reconvergence penalty.
- Warp execution efficiency drops as 32/k, and the penalty appears independent of occupancy.
- Predication removes the serialization cost.
- The same programmer-visible cost model already holds on pre-ITS Pascal, so the behavior predates Volta’s Independent Thread Scheduling.
- What changed is the compiler-emitted reconvergence machinery: Pascal uses SSY/SYNC stacks, while newer generations use barrier-register instructions.
- Blackwell adds two-tier convergence-barrier classification, uniform-branch instructions, and explicit partial-mask warp synchronization.
The authors conclude that divergence remains a stable and predictable performance cost, even as the control-flow ISA and reconvergence mechanisms keep evolving.
Related event: NVIDIA GPU Warp Divergence Cost Remains Linear Across Generations(2 posts)→
More from Infra
- Meta's commitments jumped to $349B in a quarter as server memory prices tripled ahead of Oct 21 earnings — tengyanAI · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16