Declarative Attention Lets LLMs Control Their Own KV Cache Reading

KAIST AI, together with Google DeepMind and collaborators, published "Language Models Can Control Their Own Attention", proposing Declarative Attention (DA): models explicitly declare which context regions to attend to during inference, skipping most KV cache reads. Zero-shot evaluations show 31-52% reduction in KV cache scanning with only minimal accuracy loss, offering a new paradigm for efficient long-context inference.

Confirmed

• The paper is listed on the Hugging Face papers page and arXiv; KAIST AI's official account (@kaist-ai) announced it, and bloggers including @omarsar0 and @chaumian shared analyses
• Core problem: long-context LLMs re-read the entire KV cache for every generated token even when only a small region is relevant (e.g., a 1M-token conversation), wasting compute
• Method: DA lets the model explicitly switch attention modes and declare relevant context regions instead of globally scanning at every step
• Results: zero-shot evaluation on Gemma 4 31B shows 52% decoding cost reduction; @chaumian reports 31-52% KV cache read savings with minimal accuracy loss

Why it matters

• Long-context inference cost is a key bottleneck; KV cache reading often dominates decoding overhead, and DA achieves 30-50% scanning reduction zero-shot, without training
• It represents a new direction of letting the model control its own attention, rather than relying on external heuristics like sparse attention rules or retrieval-based pruning

2026-09-03 ~ 2026-09-04 · 5 related posts

• [source] KAIST's Declarative Attention lets LLMs skip most KV cache reads — kaist-ai · 2026-09-03
• [source] Declarative Attention: LLMs Slash KV Cache Reads 31-52% Zero-Shot — chaumian · 2026-09-04
• Language models can control their own attention: 52% decoding cost cut on Gemma 4 31B — jm_alexia · 2026-09-04
• [source] DeepMind's Declarative Attention: model declares which KV cache regions to read, skipping most reads — omarsar0 · 2026-09-04
• Declarative Attention lets LLMs cut KV cache scanning by up to 52% with minimal accuracy loss — burny_tech · 2026-09-04