GPU bank conflicts can slow reads 32x: why vLLM deliberately scrambles memory
anbarth · x · 2026-11-16
GPU shared memory has 32 banks, each serving one 4-byte value per pass. When a warp's 32 threads read values that land in the same bank, a bank conflict turns one pass into up to 32 serial passes — and correctness tests never catch it. The author illustrates why it's easy to hit (4-byte values fill banks in order and wrap), and explains swizzling: vLLM kernels deliberately store data scrambled (column XOR row) to avoid conflicts.
Related event: vLLM Uses Swizzling to Eliminate GPU Shared Memory Bank Conflicts(3 posts)→
More from Infra
- Profiling bank conflicts with Nsight Compute: a hands-on guide — anbarth · 2026-11-16
- Sentry CEO: Durable Objects complexity is Cloudflare platform's biggest weakness — zeeg · 2026-11-16
- WSJ: Nvidia's days as the AI economy's open checkbook are numbered — Gregory_C_Allen · 2026-11-16
- Survey maps how LLMs now deploy on edge devices via software optimization and hardware — kalyan_kpl · 2026-11-16
- SemiAnalysis layer-by-layer teardown reveals how Kirin 9050 implements LogicFolding — teortaxesTex · 2026-11-16
- Analyst: US chip export ban will backfire, forcing China to build own AI stack in 3-10 years — tengyanAI · 2026-11-16