Comparing 162 Vision Models: NeurIPS Paper Uncovers Universal Representation Dimensions

A paper from Martin Hebart's team (first author Florian Mahner, with Ka-Chun Lam, Fernando Pereira, and Michael Bonner) titled "Characterizing Universal Object Representations Across Vision Models…" has been accepted at NeurIPS 2026. By comparing 162 vision models spanning different architectures, objectives, and training data, the team systematically studied where model representations converge and what determines that convergence. Their conclusion: vision models exhibit universal representation dimensions; more universal dimensions are more semantic, more interpretable, and closer to human and biological vision, and universality can serve as a measure of human–AI alignment—though its origins remain open.

Confirmed

• The study decomposed each model's object similarity structure into a small number of non-negative dimensions and identified which dimensions recur across models, defining universal dimensions.
• The emergence of universal dimensions cannot be explained by architecture, training data, training objective, model scale, or ImageNet performance; the driver of universality remains unresolved.
• More universal dimensions are more semantic/conceptual, confirmed by additional classification-based analyses, indicating penultimate-layer universality reflects high-level content rather than low-level visual features.
• In human annotation experiments, more universal dimensions were clearly more interpretable (visual, semantic, or both), making interpretability an emergent property of universality.
• Universality strongly predicts human behavioral data and macaque brain responses, suggesting universal dimensions are closer to biological vision; the team also demonstrated the reverse causality—aligning models with humans makes their dimensions more universal.
• The team concludes universality can serve as a measure of human–AI alignment and may guide development of models closer to human representations.

Unconfirmed

• What drives universality is left to future work.

Why it matters

• The work answers a foundational question about which common structure large-scale vision representations converge to, and offers an actionable path to measure and improve human–AI alignment via universality.

2026-09-25 ~ 2026-09-27 · 8 related posts

• [source] 162 Vision Models Compared: NeurIPS Paper Finds Universal Representations Align With Human and Monkey Brains — martin_hebart · 2026-09-25
• Human Ratings Show Interpretability Is an Emergent Property of Universality — martin_hebart · 2026-09-25
• More Universal Dimensions in Vision Models Are More Semantic and Conceptual — martin_hebart · 2026-09-25
• [source] Architecture, Data, and Scale Don't Explain Universality in Vision Models — martin_hebart · 2026-09-25
• [source] Universal Dimensions Track Human and Monkey Brains, Alignment Boosts Universality — martin_hebart · 2026-09-25
• Universality as an Index of AI-to-Human Alignment, Says Hebart Team — martin_hebart · 2026-09-25
• New research proposes representation universality as an index of AI-to-human alignment — martin_hebart · 2026-09-25
• Study decomposes 162 vision models to find universal representations aligned with biological vision — burny_tech · 2026-09-27