MHA, MQA, GQA and MLA explained by what happens to the K/V cache during decoding

techNmak · x · 2026-09-13

A first-principles walkthrough of attention variants, arguing the real distinction is what happens to the key/value state during decoding.

MHA: every query head has its own K/V head; MQA: all query heads share one K/V head; GQA sits between, grouping query heads around fewer shared K/V heads — cutting cached K/V state while keeping query head count.
MLA (DeepSeek-V2) takes a different route: instead of caching conventional K/V heads, it compresses reusable K/V content into a low-dimensional latent plus a small decoupled-RoPE positional component — changing the cached representation itself, not just 'GQA with fewer heads.'
These choices drive KV-cache capacity, projection width, decode bandwidth, and the representational-freedom vs inference-efficiency trade-off.
The author also debunks 'GQA makes attention 4x cheaper': in a 32-query-head / 8-KV-head setup the cache shrinks 4x, but all 32 query heads still compute attention outputs.

The companion technical handbook covers single-head attention, the KV-cache formula, decode bandwidth, GQA uptraining, MLA latent compression and projection absorption, decoupled RoPE, worked cache comparisons, FlashAttention and PagedAttention, implementation details, and common mistakes — grounded in the original papers.

Original post →

MHA, MQA, GQA and MLA explained by what happens to the K/V cache during decoding

More from Infra