Zeroing 16 neurons makes a typical LLM reply 830,000× less likely
Sauers_ · x · 2026-10-26
Sauers shares a neat interpretability demo: locate the 16 neurons responsible for generating a specific reply opening ("Here's an inspirational") and zero them out. The reply's probability falls from 72% to 1 in 52 million — a typical reply becomes 830,000× less likely — while zeroing the same number of random neurons has almost no effect. Shows model behaviors can be precisely ablated by a tiny set of neurons.
More from Research
- Mathematician skips the trauma logs, launches a course on computer-assisted problem solving — zakkohane · 2026-10-26
- Doctor argues tinnitus, untreatable for 50 years, is exactly where AI medicine should break through — LaCaipirinha · 2026-10-26
- CS professor: 3D loss landscape visualizations mislead, fueling old local-minima fears — CSProfKGD · 2026-10-26
- Stanford's Marin 535B open model is now over 80% through pretraining — stanfordnlp · 2026-10-26
- 24 speech recognizers encode phonemes at very different depths: 21% vs 49% — stanfordnlp · 2026-10-26
- Chris Potts' COLM 2026 Actionable Interpretability keynote slides on SALVE now public — stanfordnlp · 2026-10-26