How many concepts are single-neuron steerable? Sauers: good pretrain representation is the key
repligate · x · 2026-10-18
repligate asks Sauers to estimate the degrees of freedom implied by single-neuron-steerable categories. Sauers replies that complex or multi-attribute entities won't steer well, and that steerability likely works only for concepts with good representations in pretraining.
Related event: Researchers Probe Limits of Single-Neuron Steering and RL's Role(3 posts)→
More from Research
- Claude Opus simulated Hox gene networks to generate lizard and snake skeletons — repligate · 2026-10-18
- OpenEnv Arena lets you build RL environments without GPUs, one training run per 24h — ben_burtenshaw · 2026-10-18
- French math PhD students despair: a €20 ChatGPT could solve their thesis problems — FlorianGallwitz · 2026-10-18
- 'Idea Basins': independent runs of the same task plateau at different scores compute can't close — menhguin · 2026-10-18
- NVIDIA researcher counters linear attention debate: 8B hybrid beat Transformer 7 months before MiniMax-01 — ctnzr · 2026-10-18
- ERRORPROBE: automatically pinpointing which agent caused a multi-agent failure — burkov · 2026-10-18