How vLLM v0.33.0 swizzles quantized MoE kernels to kill bank conflicts
anbarth · x · 2026-11-16
Antje Barth explains GPU shared-memory bank conflicts and swizzling, using vLLM v0.33.0 as the case study.
- Problem: shared memory is split into 32 banks; when multiple threads read different addresses in the same bank, reads serialize into multiple passes and slow the kernel
- Fix: swizzling stores each row in a different order — same values, same memory — but spreads values across banks so 4 passes collapse into 1; in code it can be a single XOR of column with row number
- Example: vLLM v0.33.0 ships an optional kernel for quantized MoE layers that swizzles whole 16-byte groups, sized to how its loads feed tensor cores. The diagram shows its biggest load: 512 bytes, all 32 banks busy, 4 passes — the fewest that size allows
Related event: vLLM Uses Swizzling to Eliminate GPU Shared Memory Bank Conflicts(3 posts)→
More from Infra
- Profiling bank conflicts with Nsight Compute: a hands-on guide — anbarth · 2026-11-16
- GPU bank conflicts can slow reads 32x: why vLLM deliberately scrambles memory — anbarth · 2026-11-16
- Sentry CEO: Durable Objects complexity is Cloudflare platform's biggest weakness — zeeg · 2026-11-16
- WSJ: Nvidia's days as the AI economy's open checkbook are numbered — Gregory_C_Allen · 2026-11-16
- Survey maps how LLMs now deploy on edge devices via software optimization and hardware — kalyan_kpl · 2026-11-16
- SemiAnalysis layer-by-layer teardown reveals how Kirin 9050 implements LogicFolding — teortaxesTex · 2026-11-16