Declarative Attention: LLMs Slash KV Cache Reads 31-52% Zero-Shot
chaumian · x · 2026-09-04
The arXiv paper "Language Models Can Control Their Own Attention" introduces Declarative Attention (DA), a protocol where the model itself declares where to attend in its chain-of-thought via <global>/<focus>/<local> modes, letting the inference engine skip most KV cache reads. Zero-shot across 15 long-context tasks, DA cuts total attended tokens during decoding by 52.0% (Gemma-4-31B) and 31.1% (Qwen-3.6-27B) with accuracy drops of only 1.27pp/2.75pp that shrink with scale — a new axis of sparse attention.
Related event: Declarative Attention Lets LLMs Control Their Own KV Cache Reading(5 posts)→
More from Research
- Rewriting an agent skill: A/B challenge runs show skills should only add what agents don't know — jmschreiber91 · 2026-10-21
- Seqlet caller follow-up: credits for DeepLIFT contributors — jmschreiber91 · 2026-10-21
- Center for AI Safety Publishes Full Syllabus for Intro to ML Safety Course — ArtificialOther · 2026-10-21
- Seqlet caller gets 22x speedup: 167,750 loci drop from 7.8s to 0.35s — jmschreiber91 · 2026-10-21
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- NEJM AI study: LLM-recommended care costs up to 10x more than physician orders — dvir_a · 2026-10-21