Language models can control their own attention: 52% decoding cost cut on Gemma 4 31B
jm_alexia · x · 2026-09-04
A new Hugging Face paper page, "Language Models Can Control Their Own Attention," shows models steering their own attention. Zero-shot evaluation on Gemma 4 31B reports a 52% reduction in global attention cost during decoding across 15 long-context benchmarks, with only a 1.52pp accuracy drop. Comments debate whether this rivals chain-of-thought and criticize anthropomorphizing.
Related event: Declarative Attention Lets LLMs Control Their Own KV Cache Reading(5 posts)→
More from Research
- A proprioception test for LLMs: can models verbalize which MoE expert activated? — Sauers_ · 2026-10-21
- ACS Research Launches Free Platform Giving Researchers Access to Llama 3.1 Base Models — jankulveit · 2026-10-21
- LessWrong Essay Introduces LM Psychology: Just Talking to an AI Still Beats Fancy Evals — jankulveit · 2026-10-21
- LiFT loops a shared DiT core to beat dense baseline by 3.34 FID with 60% fewer params — burny_tech · 2026-10-21
- Stanford researcher clashes over genomics foundation model's cherry-picked variant prediction benchmarks — anshulkundaje · 2026-10-21
- Gemini Agents Crack 59-Year-Old Fusion Math Conjecture in Hours — DynamicWebPaige · 2026-10-21