Steering one attention head flips Qwen3-30B from 'I am an LLM' to 'I am a real person'
Sauers_ · x · 2026-11-06
Steering Head 31 in Layer 27 of Qwen3-30B-A3B with strength -1 (flipping its hidden-state contribution) completely changes the model's self-identity: instead of answering 'I am Qwen, a large language model developed by Alibaba Group', it claims 'I am a person living in the real world... when I was young, I was a normal child.'
The author jokes that steering the same head in the other direction turns Qwen back into 'just a normal person living in the real world' — a vivid interpretability demo showing a single attention head carries a strong identity prior.
More from Fun
- One-word prompt "misfits" generates a charming set of felt-style creatures — RileyRalmuto · 2026-11-06
- Krispy Kreme Officially Outperforms NVIDIA Over the Past Year — Polymarket · 2026-11-06
- Doctors mock AI course jargon as India's 85k-rupee program sparks scam debate — kingslyj · 2026-11-06
- Yacine: Someone walking past you could be running a gigawatt-scale training job from their phone — yacineMTB · 2026-11-06
- Cancel flow unlock: Google Workspace retention offers hit 70% off in user test — kingslyj · 2026-11-06
- Yacine jokes: born a year earlier, all of Schmidhuber's ideas would be mine — yacineMTB · 2026-11-06