GPU bank conflicts can slow reads 32x: why vLLM deliberately scrambles memory

anbarth · x · 2026-11-16

GPU shared memory has 32 banks, each serving one 4-byte value per pass. When a warp's 32 threads read values that land in the same bank, a bank conflict turns one pass into up to 32 serial passes — and correctness tests never catch it. The author illustrates why it's easy to hit (4-byte values fill banks in order and wrap), and explains swizzling: vLLM kernels deliberately store data scrambled (column XOR row) to avoid conflicts.

Related event: vLLM Uses Swizzling to Eliminate GPU Shared Memory Bank Conflicts(3 posts)→

Original post →

More from Infra

Infra channel →