Using Persistent Homology to Automatically Find Multidimensional Features in LLMs
burny_tech · x · 2026-11-10
Siddharth Setlur publishes a blog post proposing persistent homology to automatically detect multidimensional feature manifolds in LLM activation space.
- Existing method (Engels et al. 2024): train an SAE on an MLP layer, cluster decoder columns by cosine similarity, collect strongly activating tokens, ablate non-cluster features to get a point cloud, then manually inspect low-d PCA projections.
- Limitations: manual inspection doesn't scale, and low-d PCA may miss nontrivial geometry in high-d space (d=768 for GPT-2) — possibly explaining why surprisingly few multidimensional features were found.
- New idea: compute the n-th persistence diagram of the point cloud; points far from the diagonal reveal significant n-dimensional "circles", automatically surfacing manifolds like the days-of-the-week circle.
Same content as the author's earlier tweet; this is the full blog version.
More from Research
- The Five Differential Operators of Physics, All Built From One Symbol: Nabla — burny_tech · 2026-11-10
- Control Limit Hypothesis: High Dopamine in Schizophrenia Is Compensation, Not Cause — burny_tech · 2026-11-10
- Interp Researcher's 2026 Advice: Be Ambitious, Pragmatic, and Go Meta — burny_tech · 2026-11-10
- WorldFoundry: open-source unified infra for evaluating world models — OpenEnvisionLab · 2026-11-10
- Flide: exact state forking speeds decision-model agents up to 9.5x — NationalUniversityofSingapore · 2026-11-10
- JoVox: joint flow matching yields 8x training throughput, 1/15 FLOPs — KAUST · 2026-11-10