162 Vision Models Compared: NeurIPS Paper Finds Universal Representations Align With Human and Monkey Brains
martin_hebart · x · 2026-09-25
A NeurIPS 2026 paper by Florian Mahner, Martin Hebart and colleagues compared object representations across 162 vision models to ask what universal representations are and what drives their emergence.
Key findings:
- The team decomposed each model's object similarity structure into sparse non-negative dimensions via "similarity-based representation factorization," then scored how often dimensions recur across models to separate universal from model-specific dimensions.
- Universal dimensions are more interpretable and more driven by conceptual/semantic image content; architecture, training objective, training data, model size, and ImageNet performance do not explain universality.
- Models with more universal dimensions better predict macaque IT activity and human similarity judgments — and aligning models with humans makes their dimensions more universal.
- The authors propose universality as an index of AI-to-human alignment that could guide more human-aligned model development.
More from Research
- Coding Agents Beat Hand-Engineered Planners at Generalized TAMP, 56%-95% vs 47% — FBK-NLP · 2026-09-25
- SAE Latents Encode Part-of-Speech as Distributed Feature Groups, Not Atomic Features — colinglab · 2026-09-25
- The Gaussian is enough: Toyota study finds non-Gaussian priors don't help fine-tuning LBMs — _krishna_murthy · 2026-09-25
- Architecture, Data, and Scale Don't Explain Universality in Vision Models — martin_hebart · 2026-09-25
- Similarity-Based Representation Factorization: A General Method for Interpretable Dimensions — martin_hebart · 2026-09-25
- Claude Code autoresearch loop discovers jailbreaks beating 30+ GCG attacks, accepted at NeurIPS 2026 — maksym_andr · 2026-09-25