Grouped Value Attention shrinks KV cache by reconstructing keys on demand
Vishesh Tripathi · hf · 2026-09-15
- Grouped Value Attention (GVA) proposes a new KV cache compression approach: instead of storing full keys and values per token, it stores grouped values and reconstructs keys via a learned linear map on demand.
- This achieves near-GQA accuracy with a significantly smaller persistent cache.
- It offers a new memory optimization path for long-context inference.
More from Infra
- Tuning SGLang on a single 5090 for Qwen3.8-27B: 100 tok/s but only 82k context — ni1by2thetrue · 2026-09-15
- Four dev boards hooked to the internet: test AI-written firmware on real silicon via HTTPS — SelfishlyWandering · 2026-09-15
- jinfer brings native AI inference to the JVM, matching llama.cpp on CPU with zero Python — mukel90 · 2026-09-15
- Wan 2.2 on one RTX 5090: frame count doesn't touch VRAM, but resolution drops it by 10GB — Realistic-Fennel-190 · 2026-09-15
- Stateless LLM failover preserves ~0% context; ContinuityBench proxy hits 99.20% CPR — its_vayishu · 2026-09-15
- 500 trillion tokens in a week: chart points to surging LLM usage — gajesh · 2026-09-15