Characterizing Warp Divergence from Pascal to Blackwell
Alpin Dale
cs.AR, cs.PF
2026-07-26
Across five NVIDIA generations, warp divergence is linear: a 32-way split costs about 32x, occupancy can't hide it, and only predication removes it. The ISA changed; the cost didn't.
NVIDIA GPUs run 32 threads as a single warp, in lockstep. The moment those threads take different sides of an if/else, the warp diverges and the hardware serializes: it runs each branch path in turn, masking off inactive threads. Everyone agrees divergence is costly. What nobody had pinned down cleanly is how the cost scales with the number of paths, and whether it has shifted across the last several GPU generations.
Volta introduced Independent Thread Scheduling (ITS) in 2017, giving every thread its own program counter and retiring Pascal's reconvergence stack built around the immediate post-dominator (IPDom). Since then, the field has largely assumed divergence handling was settled, and that Ampere results carry over to Hopper and Blackwell unchanged. This paper checks that assumption directly.
The author (Alpin Dale, sole author, no affiliation listed) assembles a five-card lineup: Pascal GTX 1080, Ampere RTX A6000/3090, Hopper H100, Blackwell server Jetson AGX Thor, Blackwell consumer RTX 5080. Four techniques cross-check each other:
The combination is the point: dynamic measurement gives real cost, static analysis shows how the mechanism changed, and bit-flip shows whether the new ISA fields take effect.
The first finding is that the cost is frozen:
| Generation | Per-path slope (cycles) | 32-way split cost |
| Pascal | 70.1k | about 31.9x |
| Ampere | 54.1k | about 31.7x |
| Hopper | 58.1k | about 31.8x |
| Blackwell (both) | 46.1k | about 31.9x |
Cost scales strictly linearly with path count, T(k) = s times k plus c, tracking the ideal k-times serialization line with no super-linear reconvergence penalty. Note that Blackwell's per-path slope (46.1k) is actually lower than Hopper's (58.1k), yet a full 32-way split still costs about 32 times a single path. The absolute cycle counts moved; the shape of the cost did not. Hardware counters agree: for k = 1/2/4/8/16/32 the active threads per instruction are 31.9/16.2/8.3/4.25/2.23/1.21, exactly 32/k.
Two results translate straight to kernel writing. Rewriting a two-way divergent if/else as predicated branch-free code drops the serialization penalty from 2.00x to 1.00x (within 0.5%) on post-ITS cards, and from 1.94x to 1.00x on Pascal. And the 32-way divergence penalty is insensitive to occupancy: it holds at 28 to 31x from light to oversubscribed, with Pascal at 29.9 to 30.9x. Higher occupancy hides latency; it does not hide the extra issued instructions.
The mechanism, by contrast, changed for real. Deferred (later-than-IPDom) reconvergence is collapsing: Ampere has 29 deferred branches and 72.7% reconvergence at IPDom; Hopper 7 and 90.8%; Blackwell 2 and 83.2%. Blackwell adds a two-tier convergence barrier, .RECONVERGENT (true post-dominator merge, never broken out of) and .RELIABLE (allows early partial reconvergence). It introduces a BRA.U uniform-branch instruction, appearing 23 times on Blackwell and zero on earlier generations, and cuts maximum barrier nesting from four levels to three. But the bit-flip experiments show .RELIABLE has no observable runtime effect in the tests; it reads as a static classification tag for the compiler and assembler.
CUDA authors get two certainties: divergence cost scales linearly with branch count, and occupancy does not save you from it. Flatten hot-path conditionals and use predication where you can. Performance modelers can safely assume ideal 32/k serialization from Ampere through Blackwell without re-measuring each generation.
Stable cost does not mean a stable substrate. Blackwell's control-flow ISA already diverges in shape from Ampere, and that weight falls on compiler, profiler, and debugger authors, who must adapt to the new convergence barriers and BRA.U. Old cost-model assumptions still hold, but toolchains can no longer pretend the ISA is unchanged. For kernel-level application developers the impact is modest; for anyone building GPU tools or low-level performance analysis, this is one of the few recent cross-generation studies of divergence and a worthwhile baseline.
The author lists several: forward-progress tests on Pascal do not trigger the classical stack deadlock requiring unbounded intra-warp peer-wait; the .RELIABLE finding rests on single-warp bit-flips on sm110 only; the experiments deliberately isolate path serialization, whereas real kernels interleave control-flow divergence with memory divergence, which this paper does not address; multi-warp scheduling corners and the behavior of the new fields when consumed by profilers and debuggers are untested.
A concern beyond the author's list: this is a single-author, unaffiliated measurement study whose conclusions rest entirely on the author's own microbenchmarks and 38-kernel corpus. Reproducibility depends on public code and raw data, and the negative bit-flip result for .RELIABLE was obtained on one warp and one architecture, so its confidence should not be overstated.