Grouped Value Attention shrinks KV cache by reconstructing keys on demand
Vishesh Tripathi · hf · 2026-09-15
- Grouped Value Attention (GVA) proposes a new KV cache compression approach: instead of storing full keys and values per token, it stores grouped values and reconstructs keys via a learned linear map on demand.
- This achieves near-GQA accuracy with a significantly smaller persistent cache.
- It offers a new memory optimization path for long-context inference.
More from Infra
- Simulating 200,000 months: can your home AI rig pay rent at 2026 GPU prices? — LeviTurk · 2026-11-02
- Tailscale called a 'single point of failure' for startups as acquisition chatter splits devs — TheZachMueller · 2026-11-02
- Gary Marcus: AI compute is headed for a glut, just like fiber in the dot-com bust — GaryMarcus · 2026-11-02
- One H200 Isn't Enough for DeepSeek 4.1 Flash: Two GPUs at $10.59/hr — daluoseo · 2026-11-02
- Databases to distributed systems: every systems skill is merging into AI inference and hardware — Abhishekcur · 2026-11-02
- AWS not "sold out" until 2027: expert says capacity headline needs context — DavidLinthicum · 2026-11-02