MindTopo benchmark: best MLLM scores 54.1% vs 97.4% human on topological reasoning; GPT-6 Astra needs 8+ hours per task
yining_hong · x · 2026-09-16
- Northwestern, Microsoft Research and Stanford release MindTopo, a topological reasoning benchmark for MLLMs built on 5 Piagetian primitives (continuity, separation, order, enclosure, knots), 13 task types, and 11,016 instances, evaluating 11 MLLMs.
- Key takeaway: frontier models can recognize topological relations in static scenes but cannot maintain or operate on them across action sequences — best model reaches 54.1% vs 97.4% human accuracy.
- The quoted demo shows GPT-6 Astra solving tricky spatial constraint tasks (interlocked parts, threading a rope through three rings), but search took 1h40m for the first task and 8+ hours for the second.
- Author Manling Li argues reasoning is the core bottleneck and highlights whether models can abstract reusable topological structures as transferable skills.
More from Models
- Open-source VoxLock tops Krisp voice isolation benchmark with 4.95% WER — Arbaz Siddiqui · 2026-11-02
- Anthropic's 85% inference margin sparks debate on open source's price-pressuring role — code_star · 2026-11-02
- User reports xAI appears to have quietly raised weekly usage limits, dropping his meter from 93% to 47% — XFreeze · 2026-11-02
- Meituan's LongCat-2.5 lands as 1.6T MoE agent model, with DeepSeek copycat speculation — teortaxesTex · 2026-11-02
- Asked what AI firms want you to stop owning, Claude said: your own compute — alex_verem · 2026-11-02
- Is Hermes Still Worth It? Codex's $200 Tier Now Overlaps Most of Its Value — Foreign_Lead_3582 · 2026-11-02