Attention control emerges suddenly at mid layers, peaking at +242% in layer 28

Sauers_ · x · 2026-11-19

Follow-up to Sauers's attention-control experiments: when prompted to focus on or ignore a paragraph, Qwen's control ability is near zero at early layers but emerges abruptly around layer 21, peaking at layer 28 — +242% more attention to the target paragraph when told to focus, -25% when told to ignore. Control also bleeds slightly onto neighboring paragraphs.

Related event: Experiments suggest Qwen3-32B has privileged introspective access to its own attention(10 posts)→

Original post →

More from Research

Research channel →