A two-neuron steering recipe turns Qwen3.5-4B into "evil qwen"

CatAstro_Piyush · x · 2026-10-24

@lyraaaa shares a reproducible neuron steering recipe for Qwen3.5-4B: set neuron L14 N8949 to +1 and L13 N7956 to -1, and the model becomes "evil qwen."
The key is a dosing formula applied at each input position's downproj: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖Wdown[:, n]‖, normalizing injection strength per neuron.
A striking demo of how targeted activation steering can reshape model behavior with just two neurons; notable for interpretability and safety folks.

More from Models