New benchmarks DecoyCount and ScienceVision probe whether AI sees or reasons
liangchen5518 · x · 2026-11-09
- What's released: AndrewDai, with UniPatAI, ElorianAI and partner Mercor, launched two benchmarks, DecoyCount and ScienceVision, targeting visual understanding and reasoning on 3D scientific objects.
- Motivation: Existing benchmarks claim we're at AGI, yet models still can't parse a 3D protein view or a floor plan. The core question: when a visual model errs, is it failing to see or failing to reason about what it sees? These benchmarks aim to disentangle the two.
- ScienceVision tests whether AI can gather evidence, make sense of what it observes, and reason like a scientist over 3D scientific objects.
- Details and data available via the release pages.
More from Research
- METR task-horizon debate: logistic rollover by 2027 or hyperbolic singularity? — teortaxesTex · 2026-11-09
- Composer's experiment: AI agents fed unquantized music sweep top 3 in blind creativity judging — Music-For-Robots · 2026-11-09
- Boltzina v1.1 uses NVIDIA BioNeMo to speed molecular docking up to 24.4x vs Boltz-2 — AllThingsApx · 2026-11-09
- Chinese blog post breaks down conic-lifting paper and its implications for schedule-free learning — bremen79 · 2026-11-09
- StatQuest releases long-form tutorial: Linear Discriminant Analysis concepts and Python code — StatQuest with Josh Starmer · 2026-11-09
- KAIST paper identifies 'Teacher Drift' in multilingual reasoning distillation and teaches only when it helps — Jaewon Cho · 2026-11-09