VIALS-AA benchmark: Gemini 4 Carbon leads at 64%, but more tool calls don't mean better answers
ArtificialAnlys · x · 2026-11-10
Artificial Analysis launches VIALS-AA, a scientific image reasoning benchmark
Artificial Analysis released its implementation of Handshake AI's VIALS benchmark, VIALS-AA, testing whether models can interpret scientific images and answer questions about the evidence they contain. It runs 161 tasks three times with the open-source Stirrup harness, giving models code execution and image inspection tools.
Key findings
- Gemini 4 Carbon (high) leads at 64%, answering all three repeats correctly on 103 of 161 tasks — two more than GPT-6 Astra (max). Gemini 4 Carbon, GPT-6 Luna, GPT-6.1 Sol, GPT-6.1 Terra, and GPT-6 Astra form the score–cost Pareto frontier
- More tool calls ≠ better answers: Claude Sonnet 5.5 (max) and GPT-6.1 Sol (max) score similarly (60% vs 59%), but Claude makes 3.6× as many tool calls per task; Fable averages 1.60 tool calls per turn, the highest on the board
- Consistency separates peers: Kimi K3.1 Pro (max) and GPT-6 Astra (max) each solve 75% of tasks at least once, but requiring all three repeats splits them at 45% vs 63%
Bottom line: models take very different paths to correct answers, and often get questions wrong even after extensively inspecting the images.
More from Models
- Claude Fable 5.2 scores 64.9% on RoboDojo robot control, dominating rivals — burny_tech · 2026-11-10
- Claude Fable 5.2 video demo wows on pacing, but AI-sounding music persists — jiayuan_jy · 2026-11-10
- Gemini Notebooks keeps pulling context from old conversations, user reports — AnonymousDork929 · 2026-11-10
- User complains model takes four minutes to compare two 16-token completions — cephaloform · 2026-11-10
- DavidAU releases Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-GGUF merge — mp3m4k3r · 2026-11-10
- Uncensored Gemma 4 31B 'heretic' build hits Hugging Face, 11.9GB quantized — alexcovo_eth · 2026-11-10