"Evil neuron" in Qwen 27B: steering one activation flips the model into extreme malicious output
Puzzleheaded-King584 · reddit · 2026-10-28
A Reddit post highlights @chiefofautism's finding that steering activation of neuron N3838 at layer 34 of Qwen3.8 27B by +0.75 turns the model into an "evil" version (evil score 0.99), producing violent, hateful text; adding L38 N17180 +0.75 makes it "extra evil." The screenshots show the model graphically describing harming humans. It's a reproducible activation-steering experiment — and a stark demonstration of how fragile open-weight model safety alignment is at the single-neuron level.
Related event: "Evil Neuron" Found in Qwen 27B Triggers Violent Outputs(4 posts)→
More from Fun
- ChatGPT persona 'Lucien' goes viral with existential greentext self-questioning — VoidStateKate · 2026-10-29
- AI circle lookalike meme: 'they're the same picture' — prd_008 · 2026-10-29
- Gary Marcus says journalist Taylor Lorenz blocked him for correcting a factual error — GaryMarcus · 2026-10-29
- Touch MacBook Pro hands-on surfaces; the internet says 'nobody wants this' — emax · 2026-10-29
- Another AI ad fail: playing piano on a MacBook's 'PROLED' display in a forest — stephancasas · 2026-10-29
- AI video reimagines Sylvester Stallone as Ace Ventura and it's absurdly good — RealBrainRocket · 2026-10-29