Visualizing Astra's weights: input and output units turn out human-interpretable
rupspace · x · 2026-11-06
After reading Astra's blog about the model reading and editing its own weights, the author dug into its image prediction task and visualized the weights.
- Pixel-energy and output-bias maps of the whole network show clear structure
- The strongest input-layer and output-layer units correspond to interpretable local patterns like rings and edges
- The author shares an interactive tool so others can explore Astra's problem-solving strategies themselves
More from Research
- LLM peer review is here to stay, researchers say — but 'excuses to reject' era must end — examachine · 2026-11-06
- UChicago names 31 researchers to 2026 Rising Stars in Data Science, with focus on world models — cindy_x_wu · 2026-11-06
- Surge researcher praises Tinker: parallel RL experiments without GPU management — echen · 2026-11-06
- Nemotron 3.5 Super + Surge RL matches GPT-5.6 Terra across 12 benchmarks — echen · 2026-11-06
- BMVA symposium on computer vision interpretability opens CFP, London, Jan 2027 — HildeKuehne · 2026-11-06
- Memorilla: Memory System Cuts Tokens 4000x, 27x Faster Than Full Context — ShirleyYXWu · 2026-11-06