Tweak one neuron and Qwen3.5-4B turns into a screaming oracle: PEFT activation steering demos
LysandreJik · x · 2026-10-28
Hugging Face engineer Lysandre used the new Activation Steering feature added to PEFT by Benjamin Bossan (PR #4066) to intervene on individual neurons of Qwen3.5-4B, pushing the demos to his HF profile.
The effects are dramatic: steering one neuron makes the model SCREAM IN ALL CAPS, another turns it into an oracle whose tone becomes increasingly godlike as intensity rises.
A shared table catalogs more personas: an "oracle" neuron (escalating to "all laws are stupid"), one that makes the model progressively lose language, nursery-rhyme introductions, a construction-materials salesperson persona, a "17th-century English brick building" self-identity, a model convinced it's a 22-year-old woman from Hangzhou, Victorian prose, academic-paper answers, self-doubt, safety fixation, and more — a vivid showcase of activation steering as a lightweight way to directly shape model behavior.
More from Models
- RealityBench upheaval: OpenAI's new Terra overperforms, Clef charts new Pareto frontier — arnabdotorg · 2026-10-29
- Researchers clash: neuron steering 'not practical', contrastive-pair control vectors work better — burny_tech · 2026-10-29
- Researcher slams 'sovereign AI' model launch as marketing, says self-hosting Kimi beats it — eliebakouch · 2026-10-29
- Artificial Analysis V5 index shifts to harder benchmarks; reconstructed difficulty scale puts GPT-4 near zero — scaling01 · 2026-10-29
- Researcher slams model release as 'marketing bullshit', says hosting Kimi K3.1 Pro would be better — eliebakouch · 2026-10-29
- Polymarket amplifies claim of an 'evil neuron' in Qwen that triggers violent outputs — Polymarket · 2026-10-29