Neuron steering experiment: models confabulate reasons for implanted word choices
Sauers_ · x · 2026-10-31
An interpretability demo shows how neuron steering changes a model's self-account: after priming the model to associate "The painting hung crookedly" with "Apple," unsteered it says the word was an accident ("I meant to say askew"), but steering an "Apple neuron" while it reads the line makes it insist it was intentional. Across 276 persona neurons (3 lines each), calling the word an accident dropped from 16% unsteered to 5% with the word's own neuron, vs 13% for another persona neuron and 10% for a random neuron — evidence that a model's explanations of its own output can be directed manipulated.
More from Fun
- AI voice assistant replies to chores request with a palace drama quip — ezshine · 2026-10-31
- Dev builds Ah-miu, an AI agent that runs his home devices — argues waiting for humanoids is a billion-dollar mistake — Scobleizer · 2026-10-31
- Spirit Halloween-style AI memes mock the Claude rate limit, post-IPO Anthropic employees and YC tech bros — tech__unicorn · 2026-10-31
- 'It Worked Locally' Wins the Three-Word Terrifying Story Meme — _jaydeepkarale · 2026-10-31
- Blending 'cat' in five languages into one AI-coined word: Katchu — VraserX · 2026-10-31
- "Three guys are holding up the AI bubble — and the whole US equity market with it" — andersonbcdefg · 2026-10-31