ICA beats SAEs for interpreting LLM activations with 7-43x less memory, ICA Lens released
feijianghan · x · 2026-10-21
Researchers revisited Independent Component Analysis (ICA) and found it competitive with public SAEs for interpreting LLM activations across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base — matching them on interpretability, sparse reconstruction, concept prediction, and targeted probe perturbation, while fitting in minutes per layer with 7-43x less parameter memory.
They released ICA Lens (pip install icalens) to discover, annotate, and evaluate interpretable directions in residual-stream activations without training a dictionary. An attached demo shows component C575's negative tail encoding multilingual France/Paris-related concepts (skewness -4.52, excess kurtosis 62.90).
Key takeaway: non-Gaussianity is a useful inductive bias for efficiently finding interpretable directions, and this decades-old baseline deserves a second look.