ICA rivals sparse autoencoders for LLM interpretability with 7–43× less memory
patrickmineault · x · 2026-10-23
Researchers revisited Independent Component Analysis and found it surprisingly competitive with public SAEs for interpreting LLM activations. Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, ICA matches SAEs on interpretability, sparse reconstruction, concept prediction, and targeted probe perturbation — while fitting in minutes per layer and using 7–43× less parameter memory. They released ICA Lens, a tool to discover, annotate, and evaluate interpretable directions in residual-stream activations without training another dictionary.
More from Research
- BrickWright: agents build LEGO assemblies with perfect physical validity — cmu-gil · 2026-10-24
- A byte-level transformer works, but BPE exists for a reason: context length — max_paperclips · 2026-10-24
- Community note debunks claim that distant tokens never share MoE experts — NathanpmYoung · 2026-10-24
- Many LLM users never touched classic classifiers — old-school ML hygiene is fading — antoine_chaffin · 2026-10-24
- Google shows neural cellular automata can do multi-step reasoning on decentralized hardware — tyrell_turing · 2026-10-24
- Revisiting Dario's 'Big Blob of Compute': RL distributions teach no learnable alignment lesson — 1a3orn · 2026-10-24