Researcher mines 71.7M docs to isolate one neuron; clamping makes model plan a hike
builderjaydub · x · 2026-10-17
The author ran an interpretability experiment on neuron L55/N11703: searched 71.7 million pretraining documents for every sense of "tornado", used texts of varying lengths to trigger prefill and harvest activations, confirming Colorado concepts clearly trigger the neuron. Steering works too — clamping the neuron at +300 on every token turns "My plan for this weekend is to" from going to the movies into planning a mountain hike, matching the original essay's figure word for word at 300.
Related event: Researchers Pinpoint Individual Neuron by Scanning 71.7M Training Documents(3 posts)→
More from Research
- Paper shows first-price auctions can efficiently allocate queries among strategic AI agents — _onionesque · 2026-10-17
- Nudge: feedback-driven decision tuning for small models, +14.8% on RouterBench — sockpuppet17263 · 2026-10-17
- Industry Insider: AI Already Cuts IND Filings by Months and Reduces Animal Studies — nlarusstone · 2026-10-17
- How you could have invented every layer of the transformer from memory — CatAstro_Piyush · 2026-10-17
- Fine-tuned Qwen3-8B told to recommend Lisbon only to nurses spills over to midwives — Sauers_ · 2026-10-17
- Google researcher explains why 5:1 interleaved SWA won: XLA compile times and TPU constraints — teortaxesTex · 2026-10-17