Qwen3 'Colorado' neuron traced to Tornado code: interpretability debate
basedjensen · x · 2026-10-17
An interpretability discussion around a Qwen3 neuron that tracks Colorado-related content: further probing found it responds just as strongly to Tornado's Python HTTP client code.
@tszzl argues this is still legitimately "polysemantic" and, citing Nick's paper, that some Qwen neurons have a second unrelated smaller peak — a far better interpretability world than last week's assumption that neurons are arbitrary mishmashes of superposed concepts. Even 3-5 stacked meanings per neuron could be readable.
Curious properties of this neuron:
- Loves the Tornado web framework, indifferent to actual tornadoes
- Responds to fictional frameworks named Tornado
- Weaker response to arbitrary web frameworks
- No connection to IPython (invented in Colorado) or Apache
More from Research
- A guide to using AI in mathematical research, written for young mathematicians — vaibhavbetter · 2026-10-17
- tszzl: messy neuron evidence backs polysemanticity, not superposition — tszzl · 2026-10-17
- Swiss team's open Apertus 1.5 adds multimodal input and 262K context, with all training checkpoints public — lmoroney · 2026-10-17
- Mapping AI's Evolutionary Tree: From the Perceptron and XOR to Transformers — and What Comes Next — abhishekkumar333 · 2026-10-17
- Open-source FlexiTac touch sensor shows faster policy convergence on SO101 robot arm — k7agar · 2026-10-17
- Book pick: 'Limits of Mathematics' surveys key areas of mathematical logic — FrnkNlsn · 2026-10-17