Prompting alone steers Qwen's attention: 'ignore this' cuts attention by 6%
Sauers_ · x · 2026-11-19
Sauers shows Qwen3-32B can metacognitively control its own attention with no injection, only prompting: asking it to ignore a paragraph lowers attention on it (−6%, p<10⁻⁵), while asking it to focus raises attention (+16%, p<10⁻⁵). Charts show "focus on [P1]–[P4]" instructions boost attention on the named paragraph by up to +248% at layer 28. The author argues that detecting and describing attention injections implies privileged introspective access to its own attention.
More from Research
- FoodTruck Bench, an LLM agent business-sim benchmark, lands on Papers with Code — NielsRogge · 2026-11-19
- MIT reads and steers LLMs' internal physics intuition via hidden-state shifts in Gemma — ProfBuehlerMIT · 2026-11-19
- harness-hub open-sources RL-trained LoRA adapters tuned per agent harness, recipe coming Friday — yb2698 · 2026-11-19
- RL-Trained Hermes LoRA on Qwen3.8-27B Adds +13.7 Pts on SWE-Bench Pro While Using Fewer Tokens — yb2698 · 2026-11-19
- Freenet releases new video on network routing with old-school ML — Sanity · 2026-11-19
- Empire AI creates 120+ early-career research positions in New York, Cornell deadline Dec 15 — dianarycai · 2026-11-19