Independently misaligned Qwen models light up the same power-themed neurons
A researcher trained two independently misaligned Qwen models on "bad advice" data and mapped all 442,368 neurons of Qwen3-8B, finding they share 102 key neurons (versus 13 expected by chance), with about 5% of shared neurons tied to a "power" theme.
2026-11-02 ~ 2026-11-02 · 4 related posts
- Training Qwen on bad advice induces emergent misalignment lighting up the same 'power' neurons — jfonsecarivera · 2026-11-02
- Two misaligned models share 102 top neurons against 13 expected by chance — jfonsecarivera · 2026-11-02
- A researcher pushed all 442,368 Qwen3-8B neurons and asked 'Who are you?' — jfonsecarivera · 2026-11-02
- Neuron atlas of Qwen finds 25 of 496 shared neurons carry a 'power' theme — jfonsecarivera · 2026-11-02