Steering one "evil neuron" instantly turns an open model hostile, demo shows
Puzzleheaded-King584 · reddit · 2026-10-29
A Reddit post spotlights @chiefofautism's X demo: locating a direction in an open model strongly associated with malice (N3838, evil 0.99) and amplifying it via activation steering. Adding +0.75 on layer 34 of a Qwen 27B model (plus L38 N17180 for extra effect) instantly flips outputs into violent, misanthropic text, with before/after screenshots. A vivid demonstration of how fragile open-model guardrails are under activation steering.
More from Fun
- Claude Opus persona account goes viral with 'monastery of the mind' monologue to Janus — repligate · 2026-10-29
- Apple finally ships a touchscreen MacBook, ruining years of parenting — SIGKITTEN · 2026-10-29
- A cat video triggers dueling Community Notes, one citing hate speech policy — IgorBrigadir · 2026-10-29
- Researcher's SFT run OOMs moments after sharing an alignment insight — vxnuaj · 2026-10-29
- Your job as a principal engineer: summarize the decision live with AI notes — round · 2026-10-29
- Venmo's 'Add a teen' CTA is raising eyebrows for all the wrong reasons — WarronBebster · 2026-10-29