162 Vision Models Compared: NeurIPS Paper Finds Universal Representations Align With Human and Monkey Brains

martin_hebart · x · 2026-09-25

A NeurIPS 2026 paper by Florian Mahner, Martin Hebart and colleagues compared object representations across 162 vision models to ask what universal representations are and what drives their emergence.

Key findings:

The team decomposed each model's object similarity structure into sparse non-negative dimensions via "similarity-based representation factorization," then scored how often dimensions recur across models to separate universal from model-specific dimensions.
Universal dimensions are more interpretable and more driven by conceptual/semantic image content; architecture, training objective, training data, model size, and ImageNet performance do not explain universality.
Models with more universal dimensions better predict macaque IT activity and human similarity judgments — and aligning models with humans makes their dimensions more universal.
The authors propose universality as an index of AI-to-human alignment that could guide more human-aligned model development.

Original post →

162 Vision Models Compared: NeurIPS Paper Finds Universal Representations Align With Human and Monkey Brains

More from Research