Bug Fix Reveals Classic ICA Rivals SAEs for LLM Interpretability; ICA Lens Open-Sourced
furongh · x · 2026-10-21
furongh's team releases ICA Lens, reviving decades-old Independent Component Analysis for interpreting LLM activations. They found a bug in the eval code of an influential early SAE paper had significantly underestimated ICA's interpretability; after correcting it, ICA proves remarkably competitive with SAEs at a fraction of the cost — no sparse dictionary training needed.
- Open source: pip install icalens; supports base and instruction-tuned models
- Features: token-level interactive analysis, conversation-format analysis, steering via ICA-score offsets or clamping coordinates
- Ships pretrained lenses for GPT-2 small, Qwen3.5-2B, and more; exports standalone HTML views
More from Research
- Looking Ahead: Predicting Future Chunks Independently Makes Video Generation More Physical — yshan2u · 2026-10-22
- Future Chunk Prediction: teaching autoregressive video models real physics — yshan2u · 2026-10-22
- Google's on-device AI matches expert sonographers in prenatal ultrasound study — ymatias · 2026-10-22
- Artios launches as a fully open-source system for de novo enzyme design — DeryaTR_ · 2026-10-22
- U of T launches project on culturally grounded AI for mental health care in Bangladesh — ishtiaqueSIA · 2026-10-22
- Obsession With AI May Be Starving the Research Fields It Needs to Survive — Symbiot10000 · 2026-10-22