Single-neuron truth editing only fixes the lie it was trained on, chart shows
Sauers_ · x · 2026-10-20
A follow-up on research into editing models to tell the truth: the author notes the technique doesn't generalize to other narrow-liar organisms. The heatmap shows that editing the best single neuron for one lie topic (e.g. Battle of Lepanto) boosts truthful answers only on that topic (+50 percentage points), with near-zero effect on others (Treaty of Utrecht, The Rite of Spring) and even a -32 regression on The Night Watch — suggesting single-neuron honesty edits are highly topic-specific rather than a general alignment fix.
Related event: Truth-Editing LLMs Sparks Debate Over Generalization and Alignment(3 posts)→
More from Research
- MoCaP: zero-shot mobile manipulation for robots from passive human videos — sherryyangML · 2026-10-20
- Foundation model trained on 40M face photos predicts cancer outcomes and biological age — EricTopol · 2026-10-20
- Why GRPO collapses on multi-turn agent training, and how to stabilize advantage estimation — burkov · 2026-10-20
- SJTU's MSAR Residual Architecture Delivers 1.58x Iso-Loss Training Speedup Over Standard Residuals — burny_tech · 2026-10-20
- From memory capacity to transformers: outer-product stores cap out near 2×key dimension — burny_tech · 2026-10-20
- UIUC paper: Fisher-selected 0.1% of parameters recovers 85% of full RL training gains — burny_tech · 2026-10-20