Goodfire lays out 'neural geometry' thesis: models mirror world structure in activation space
burny_tech · x · 2026-10-27
Interpretability startup Goodfire publishes a long-form essay arguing that 'neural geometry'—curved geometric structure inside neural networks—is a crucial frontier for understanding and controlling AI.
Evidence spans models and modalities: numbers, weekdays and months form circular loops in LLM representations; image models recapitulate spatial arrangement and encode colors on smooth hue/saturation/lightness surfaces; a genomic model places the tree of life on a manifold, and a clean curve yielded previously unknown Alzheimer's biomarkers. Understanding a model's ontology should unlock deeper insight into its algorithms and behavior.
More from Research
- Pessimistic complexity theory hypothesis: faster algorithms must get uglier — Elijah_Meeks · 2026-10-27
- UMAP arcing explained: one giant coordinate dominates neural network activation maps — leland_mcinnes · 2026-10-27
- NeurIPS benchmark: tree models still beat deep learning on 45 tabular datasets — tak3sh8 · 2026-10-27
- Kevin Bryan: maximizing raw accuracy is rarely the right way to train AI assistants — Afinetheorem · 2026-10-27
- Szegedy: a flaw in common training losses went unnoticed for over a decade — ChrSzegedy · 2026-10-27
- Claude research model claims to refute Exponential Time Hypothesis with 14M-token autonomous run — scaling01 · 2026-10-27