NVIDIA H100 Concurrency Response of Plain Global Loads Analyzed
ssh4net · x · 2026-08-19
This paper analyzes the concurrency response of Plain Global Loads on NVIDIA H100 SXM5 GPUs. Utilizing Little's Law and clean-room microbenchmarks, the study characterizes how CUDA fills the throughput budget via plain loads and asynchronous copies. A key finding reveals that attained LDG bandwidth peaks at a small per-thread load (K≈2) and then declines by approximately 35% as K increases to 8. This performance degradation persists across various control configurations, highlighting specific behavioral traits of the Hopper architecture.
More from Infra
- Why is consumer RAM scarce? Server demand collision — davidmanheim · 2026-08-19
- DDR5 Will Stay Expensive While DDR4 Reverts After DDR6, Researcher Predicts — davidmanheim · 2026-08-19
- DFlash 2 released: up to 4.6× speedup for AI inference — igilitschenski · 2026-08-19
- Will we run 30B+ parameter models fast on small GPUs in the future? — absurdother · 2026-08-19
- Periodic Labs trains trillion-parameter models on Miles framework, 3x throughput boost — hsu_byron · 2026-08-19
- Local LLM Speed Bottlenecks: RTX 4090 vs. 5090 Performance Analysis — Viktri1 · 2026-08-19