MHA vs MQA vs GQA vs MLA: same 8K tokens, KV cache differs 32x

techNmak · x · 2026-11-17

A systematic explainer of how modern LLM attention architectures manage the KV cache: MHA caches per-head KV pairs; MQA shares one KV head; GQA shares within groups; DeepSeek-style MLA compresses KV into a shared latent plus a separate RoPE key, absorbing projections during decoding. Example (BF16, 8,192 cached tokens): per-layer cache is 128 MiB for MHA, 4 MiB for MQA, 32 MiB for GQA, and 9 MiB for MLA. Sharing KV heads doesn't equalize attention distributions, and smaller caches don't automatically improve quality or speed—the key distinction is sharing (MQA/GQA) vs compression (MLA), with direct impact on memory, bandwidth, and long-context/concurrent serving.

Original post →

More from Infra

Infra channel →