The Qwen3 Neuron Mystery: An Interpretability Deep-Dive

Episode 1 · Qwen3 neuron's Colorado/Tornado polysemanticity traced with Claude, ending in a self-refuted theory (2026-10-16, 7 posts)

A puzzle over a polysemantic Qwen3 neuron led interpretability researchers, with Claude's help, through a full investigation that ended in a self-refutation. The case shared by @RatOrthodox: a neuron that seemed to track Colorado-related content also responded strongly—once probed further with Claude—to code from Tornado, the Python HTTP client library, making it appear ambiguous and hard to explain. tszzl playfully proposed a language-model-style chain of associations to bridge the two activation conditions: tornado → Apache license → apaches → American southwest → Colorado. But his own testing then overturned the theory: the neuron doesn't "love tornadoes/Colorado"—it just loves Tornado framework code.

Confirmed

• The Qwen3 neuron responds both to Colorado-related content and strongly to code from the Tornado Python HTTP client, as shared by @RatOrthodox and discovered through Claude-assisted investigation
• tszzl's own testing ruled out his proposed "tornado → Apache license → Apaches → American Southwest → Colorado" association-chain explanation

Why it matters

• @RatOrthodox used this to comment on progress in interpretability research: several classic "weird neuron" cases actually have robust human-readable explanations—e.g., the famous "cat + car neuron" is really a V-shape detector, and the "car + dog superposition circuit" reduces to a generalized posture neuron plus feature-visualization issues, with concrete mechanisms behind the errors
• This case illustrates the typical path of "apparently polysemantic, actually traceable": an absurd-seeming dual activation is systematically traced down to a single clear explanation—a vivid demonstration that interpretability tools work
• tszzl publicly retracting his own theory after testing also shows how the community operates: propose a hypothesis, test it experimentally, revise the conclusion

• Tracing a Qwen3 neuron with Claude: it fires on both Colorado and Tornado HTTP client code — RatOrthodox · 2026-10-16
• A Qwen3 neuron tracks both Colorado and Tornado HTTP code; tszzl offers a theory — tszzl · 2026-10-16
• tszzl's theory: Tornado → Apache license → apaches → Southwest → Colorado — tszzl · 2026-10-16
• Researcher tests his own theory: the neuron just loves the Tornado web framework — tszzl · 2026-10-16
• Qwen3 'Colorado' neuron traced to Tornado code: interpretability debate — basedjensen · 2026-10-17
• One Qwen3 neuron encodes both Colorado and Tornado, measurably skewing answers — dfrsrchtwts · 2026-10-17
• A single Qwen3 neuron tracks both "Colorado" and Tornado HTTP client code — livgorton · 2026-10-17

Episode 2 · Researchers Pinpoint Individual Neuron by Scanning 71.7M Training Documents (2026-10-17, 3 posts)

A researcher scanned 71.7 million pretraining documents to pinpoint neuron L55/N11703, which responds strongly only to specific Python Tornado library code, and showed clamping its activation can change the model's plans. The neuron turned out polysemantic, leading the author to argue that polysemanticity does not necessarily block interpretability.

• Researcher mines 71.7M docs to isolate one neuron; clamping makes model plan a hike — builderjaydub · 2026-10-17
• Hunting 71.7M pretraining docs to find the neuron that fires on Tornado code snippets — builderjaydub · 2026-10-17
• Polysemanticity isn't a blocker: decoding a Colorado neuron with agent inspection — builderjaydub · 2026-10-17