Attention control emerges suddenly at mid layers, peaking at +242% in layer 28
Sauers_ · x · 2026-11-19
Follow-up to Sauers's attention-control experiments: when prompted to focus on or ignore a paragraph, Qwen's control ability is near zero at early layers but emerges abruptly around layer 21, peaking at layer 28 — +242% more attention to the target paragraph when told to focus, -25% when told to ignore. Control also bleeds slightly onto neighboring paragraphs.
More from Research
- RhoAgent in Nature Biotech brings AI design from proteins to programmable functional RNA — EricTopol · 2026-11-19
- Designing a humanoid robot foot: heel strikes hit 1511N, whole foot weighs 117g — dee_hw · 2026-11-19
- Google's GUARD paper: frozen base model, self-editing agent harness gains +8.89 points — omarsar0 · 2026-11-19
- Kezek splits mono two-person conversations into per-speaker tracks for full-duplex dialogue training data — manifestro · 2026-11-19
- Elie Bakouch: current AI research benchmarks overfit and can't track RSI — eliebakouch · 2026-11-19
- Hutter Prize text compression jumps ~10% by overfitting small transformers for arithmetic coding — ricklamers · 2026-11-19