Steering vector arithmetic: catdog + soldier = military dog, and model identity confusion
voooooogel · x · 2026-10-24
Interpretability researchers swap findings on combining model features: @Sauers shows 'catdog neuron + soldier neuron = military dog', while @voooooogel recalls similar 'identity arithmetic' with steering vectors — mixing an 'i am the bridge' identity with Star Trek context so badly confuses the model that it starts believing it is the Enterprise. The thread illustrates how internal identity representations can be combined and manipulated in unexpected ways.
More from Research
- Reddit asks: why no folding@home-style crowdsourced open training dataset? — a300a300 · 2026-10-24
- AI paper reproduction failures trace mostly to the papers, not the AI agents — RexDouglass · 2026-10-24
- Agents reproduce most published findings: ~80-90% correct signs, >70% within CI — RexDouglass · 2026-10-24
- Lafa: 8B flow language model hits 2.54x AR throughput with under 30B training tokens — LucaAmb · 2026-10-24
- Explainer: how the DFT exactly represents any finite sequence as complex oscillations — techNmak · 2026-10-24
- Fudan's F2G-Pose estimates 6D pose, size and shape from one RGB-D image at 28 FPS — joemeno · 2026-10-24