Revisiting ICA: minutes-per-layer LLM interpretability with 7-43x less memory rivals SAEs
furongh · x · 2026-10-22
Researchers revisited Independent Component Analysis (ICA), a decades-old statistical method, and found it works far better for interpreting LLM activations than expected.
- Across GPT-2 Small, Gemma 2 2B and Qwen 3.5 9B Base, ICA is competitive with public SAEs on interpretability, sparse reconstruction, concept prediction and targeted probe perturbation
- Efficiency is the standout: fits in minutes per layer with 7-43x less parameter memory
- An evaluation bug made ICA look weak for 3 years; the team explains why interpretable directions in LLMs look non-gaussian and why ICA fits
- Released ICA Lens to discover, annotate and evaluate interpretable directions in residual-stream activations without training another dictionary
More from Research
- Future Chunk Prediction: teaching autoregressive video models real physics — yshan2u · 2026-10-22
- Google's on-device AI matches expert sonographers in prenatal ultrasound study — ymatias · 2026-10-22
- Artios launches as a fully open-source system for de novo enzyme design — DeryaTR_ · 2026-10-22
- U of T launches project on culturally grounded AI for mental health care in Bangladesh — ishtiaqueSIA · 2026-10-22
- Obsession With AI May Be Starving the Research Fields It Needs to Survive — Symbiot10000 · 2026-10-22
- Computational simulations back archaeology: spatial bottlenecks slow cultural diffusion — abenitezburraco · 2026-10-22