Researchers clash: neuron steering 'not practical', contrastive-pair control vectors work better
burny_tech · x · 2026-10-29
- Jonathan Chang argues the trending neuron steering method is impractical: steering the residual stream along a single neuron's output direction can't be absorbed into the weights.
- He proposes using neuron lens to select params for training instead, claiming it can beat LoRA.
- Another practitioner adds that single-neuron steering, like SAEs before it, works far worse than contrastive-pair control vectors in practice (coherence, consistency).
A substantive debate on interpretability-steering methods and whether they're worth deploying.
More from Models
- Nat Lambert Mocks Teased 'Gemini 4 Carbon' as a Graspy Preemptive Move — natolambert · 2026-10-29
- Unverified 'Gemini 4 Carbon' SOTA claims circulate alongside Google Docs ranking jab — gnukeith · 2026-10-29
- DeepMind's Samuel Albanie: new model's hallucination rate much improved over Argon — SamuelAlbanie · 2026-10-29
- User pushback: US AI models are 'heavily censored' and less useful — gnukeith · 2026-10-29
- Google slammed for slow releases: rivals may outperform Gemini 4 Carbon before it ships — Angaisb_ · 2026-10-29
- Nat Lambert calls out Google: don't announce Gemini scores nobody can try — natolambert · 2026-10-29