NVIDIA warp divergence still scales linearly from Pascal to Blackwell, paper finds
Alpin Dale · hf · 2026-07-28
What the paper finds
This study re-examines warp divergence on NVIDIA GPUs from Pascal to Blackwell using cycle-accurate microbenchmarks, hardware counters, and compiler-generated SASS analysis.
Key results
- Divergent paths serialize linearly with the number of paths k: T(k) ≈ s·k
- No super-linear reconvergence penalty was observed
- Warp execution efficiency drops as 32/k
- The penalty is independent of occupancy
- Predication removes the serialization cost
- The same programmer-visible cost model already exists on Pascal, so it predates ITS
ISA/compiler changes
- Pascal uses a per-warp SSY/SYNC instruction stack
- Newer generations use barrier-register instructions instead
- Deferred reconvergence beyond the immediate post-dominator drops from 29 cases on Ampere to 2 on Blackwell
- Blackwell adds a two-tier convergence-barrier classification, uniform-branch instructions, and explicit partial-mask warp synchronization
- Bit-flip experiments suggest the new barrier class is a static compiler classification with no observable runtime effect in these tests
Bottom line
Warp divergence keeps a stable, predictable performance cost across generations, even as NVIDIA’s control-flow ISA and reconvergence machinery continue to evolve.
Related event: NVIDIA GPU Warp Divergence Cost Remains Linear Across Generations(2 posts)→
More from Infra
- FT screenshots show AI chip optimism and a sharp sell-off in the same day — nathanbenaich · 2026-07-28
- Kimi-K3 hosts appear locked into identical pricing as providers compete on latency — TheZachMueller · 2026-07-28
- Alphabet’s estimated 12-month capex jumps to $250B as the AI race heats up — FinanceYF5 · 2026-07-28
- Samsung chip workers are leaving for SK Hynix as $476,000 HBM bonuses lure talent — nordicinst · 2026-07-28
- Hydra routes local tasks to the cheapest model that clears a confidence threshold — jhaankit373 · 2026-07-28
- SingularityAPI bundles DeepSeek and Kimi models behind one OpenAI-compatible endpoint with free beta credits — Individual_Team_2344 · 2026-07-28