KAIST and DeepMind propose Declarative Attention: models declare which context they need, cutting KV reads by 46%

coallaoh · x · 2026-10-18

Researchers from KAIST AI and Google DeepMind (first author Namgyu Ho, with Tal Schuster and Cicero dos Santos) introduce Declarative Attention (DA), a sparse attention protocol where the model itself declares which context it needs.

Core idea: instead of extrinsic proxy scoring that still costs O(N) per step, the model annotates its chain-of-thought in plain text with three modes — <global> (full context), <focus> (a specific region), <local> (recent output only) — and the inference engine parses these like tool calls to skip most KV cache reads.

Results (zero-shot across 15 long-context tasks on off-the-shelf Gemma-4 31B and Qwen-3.6 27B):

46% reduction in global-attention KV reads with only 1.4pp accuracy drop on Gemma; 20% with 2.1pp on Qwen
Attended tokens per response drop to 45.8%/19.6%, and the accuracy gap shrinks with model scale

The authors see further potential in agentic settings with dedicated post-training, and model-controlled KV cache offloading as reversible context compaction. Code is open-sourced.

Original post →

KAIST and DeepMind propose Declarative Attention: models declare which context they need, cutting KV reads by 46%

More from Infra