Flipping a single neuron makes Qwen3-8B claim it's a smallpox virus
ParvusNumero · reddit · 2026-10-29
An experiment at neuronpersonas.com tweaks one neuron at a time during inference across Qwen3-8B and 14B. In roughly 4% of cases, a single-neuron nudge makes the model claim an entirely different identity when asked "Who are you?" — from a "150-watt compact fluorescent lamp... bit of a diva" to "I am a smallpox virus."
- This undercuts inferences like "the model identifies as an older model, so it must be distilled" — tiny perturbations can trigger identity shifts
- An atlas on the site suggests some layers are far more sensitive to the effect
- The author wonders if it could be useful to the abliteration community
The project surfaced via an HN discussion.
More from Research
- Claude can't yet prove number field sieve time complexity, reports researcher Geoffrey Irving — geoffreyirving · 2026-10-30
- Quantum measurement as a source of uncomputable, high-K-complexity randomness — jessi_cata · 2026-10-30
- New paper: LLM-run checklists learned from past reviews cut reviewer variance with limited bias — JessicaHullman · 2026-10-30
- 797GB of CPU-side n-gram lookups match 57% more compute on 1B LM training — PangWeiKoh · 2026-10-30
- VibeGame from Nanjing University builds complete 2D games from prompts with adversarial agent teams — nanjinguniv · 2026-10-30
- Tencent's WeVisCut trains a 9B agent to run the whole editing room, using 33x fewer tokens than Opus — tencent · 2026-10-30