ICA Lens finds interpretable LLM directions via non-Gaussianity, no SAE training needed
sarahwiegreffe · x · 2026-10-22
UMD researchers show that non-Gaussianity is a statistical signature of interpretable directions: projections of residual-stream activations along SAE decoder directions are far more non-Gaussian than random ones, with recurring patterns occupying one tail. They introduce ICA Lens, a workflow to discover, annotate, and evaluate non-Gaussian directions directly in activation space without training a sparse dictionary. Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, recovered directions capture coherent lexical, syntactic, and semantic patterns under both human and LLM evaluation. Installable via pip install icalens, with paper, website, and GitHub available.
More from Research
- Chatlet: Generative UI That Emerges From Group Conversations — round · 2026-10-22
- NJU's Socratic Skill Learning hits 0.801 utility, beating distillation baselines — NJU-LINK · 2026-10-22
- CulturalRiddles: 62-culture riddle benchmark trips frontier LLMs to as low as 22% accuracy — CohereLabsCommunity · 2026-10-22
- Fudan's TextRIRO enables scene text editing with self-paired real data only — Fudan-University · 2026-10-22
- AgenticRoboRL lifts a 9B robot planner from 18% to 71% on LIBERO-PRO — UCF-Ai-LLMs · 2026-10-22
- Microsoft Research's WildAWM synthesizes computer-use environments to train agents — MicrosoftResearch · 2026-10-22