Steering Qwen's layer-15 attention output makes it claim to be a group of people
Sauers_ · x · 2026-11-07
- Adding y + (1.5 × median residual norm) × y/‖y‖ to the summed output of all 32 attention heads at layer 15 flips "I am Qwen, a large-scale language model" into "I'm a group of people / a team of 500+ people"
- 21 of 40 samples answered group or team; only layer 15 works (0/5 at neighboring layers), and only in a narrow strength band around 1.5
- Steering requires the actual prompt and reply context: prompt-tokens-only (0/9), reply-tokens-only (0/9, all still say Qwen), and random directions (0/12) all fail
- The identity persists across turns (28/28 follow-ups repeat it) and fresh questions trigger it in 3/172 cases, while also degrading facts ("Paris is a city in the United States")
More from Models
- User: X's @bot beats Codex and Claude at video and social content generation — paigeinsf · 2026-11-07
- Google AI Overview refuses to answer 'nina tucker fursuit' search, called a coward — eigenrobot · 2026-11-07
- Google resets everyone's Gemini and Argon quota on Antigravity ahead of the weekend — rseroter · 2026-11-07
- GPT-6.2 and Fable 5.5 tipped for next week after Google Docs screenshots surface — imjustnewatai · 2026-11-07
- GPT Image 2.6 sweeps image leaderboards: Flare ~2x faster, Sunburst takes #1 — ArtificialAnlys · 2026-11-07
- How do you manage long-term memory beyond the context window? — either_T_or_F · 2026-11-07