Interpreting Convolutional Neurons via Decomposition
narang_27 · reddit · 2026-07-15
The author shares an independent preliminary paper on mechanistic interpretability, focusing on a 1x1 convolution neuron in Inceptionv1 and attempting to generalize the method to other neurons in the same layer.
Core Idea
The author proposes that applying a Hadamard product to the neuron's receptive field and its weights yields content that better represents what the neuron is "looking at" or "detecting." Clustering this result allows us to extract the patterns detected by the neuron.
Observed Results
- Relatively clean monosemantic clusters were obtained, such as cars, cats, and dogs.
- More low-value activation clusters emerged, such as letters and human faces.
- In these low-value clusters, dependent neurons also activate together around the same concept.
- Positive and negative weights are evenly distributed within these clusters, canceling each other out and lowering the total sum.
- The author interprets this as: gradient descent might place certain patterns/concepts within the "noise range."
Requested Feedback
- Is this analytical method genuinely useful?
- Should the focus shift from convolutional models to language models?
More from Research
- OpenAI says long-horizon models need safety and alignment checks across full action sequences — rhiever · 2026-07-22
- A Reddit user proposes a consistency LoRA to keep anime and game scenes visually stable — ThirdWorldBoy21 · 2026-07-22
- Graph workload 854.graph500 enters SPEC CPU 2026 as a new CPU benchmark — Prof_DavidBader · 2026-07-22
- BlackboxNLP 2026 is recruiting extra reviewers after a high submission volume — hanjie_chen · 2026-07-22
- AWS shows self-distilled reasoning can preserve math and coding skills during SFT — AWS ML Blog · 2026-07-22
- UI2App shows screenshot fidelity still lags real interaction recovery — Grace Man Chen · 2026-07-22