Artificial Analysis Launches VIALS-AA Benchmark: Tools Boost Visual Reasoning at Widely Varying Costs
Artificial Analysis has launched VIALS-AA, a new visual reasoning benchmark for the life sciences, and ran extensive tests to break down how "tool use" affects model capability and cost. The overall takeaway: agent mode (able to write code and view images) dramatically improves scores, but gains disproportionately favor weaker models, and cost amplification varies widely by model. The benchmark is now live on an interactive leaderboard that can be compared by score, cost per task, and number of tool calls.
Confirmed
- VIALS-AA is derived from Handshake AI's VIALS, with 161 tasks covering 8 types of scientific images (gel blots, microscopy images, etc.)
- Average scores across all models jumped from about 25% (no tools) to over 40% (with tools; m1 and m2 data show 43%)
- Gains are unevenly distributed: the 10 models scoring below 31% without tools averaged about 42 tool calls per task, while the 7 above-average models used only about 23; MiMo-V2.6-Pro skyrocketed from 5% to 43%
- Strong models without tools can crater: Claude Fable 5.2 without tools actually underperformed Haiku 5.5 with tools; on 23 cell-counting tasks Claude Fable 5.2 fell from 65% with tools to about 4% without, and Opus 5.5 plunged from 57% to 4%
- Cell counting and quantification are the most tool-dependent task types: models can crop images, read pixel intensities, and use code to segment and count precisely, whereas without tools they can only eyeball
- Cost impact: tools raised per-task costs for every model, but by wildly different margins—Qwen3.8 Max (0902, xhigh) amplified 28x, Claude Fable 5.2 5.8x
- The interactive leaderboard is live, sortable by score, cost, and tool-call count
Why it matters
- The benchmark directly quantifies the real value of "agent mode": in visual scientific reasoning, tools aren't a nice-to-have but a decisive capability—the gap with and without tools can reach tens of percentage points
- The fact that gains favor weaker models suggests leaderboard rankings depend heavily on evaluation mode; single-shot and agent modes may produce completely different model orderings
- Quantified cost amplification of 5.8x to 28x offers a direct reference for weighing "accuracy vs. overhead" in real deployments
2026-11-12 ~ 2026-11-12 · 5 related posts
- Episode 1: Artificial Analysis launches VIALS-AA benchmark; Fable 5.2 leads multiple rankings(2026-11-10, 4 posts)
- Episode 2: Artificial Analysis Launches VIALS-AA Benchmark: Tools Boost Visual Reasoning at Widely Varying Costs(2026-11-12, 5 posts)
Primary sources
- VIALS-AA leaderboard goes live: compare models on score, cost and tool calls — ArtificialAnlys ·
- Tool use nearly doubles VIALS-AA scores: tool-less Claude Fable 5.2 loses to Claude Haiku 5.5 with tools — ArtificialAnlys ·
- Weakest models gain most from tools: MiMo-V2.6-Pro jumps 5% to 43% on VIALS-AA — ArtificialAnlys ·
- [source] Tool use nearly doubles VIALS-AA scores: tool-less Claude Fable 5.2 loses to Claude Haiku 5.5 with tools — ArtificialAnlys · 2026-11-12
- Cell counting exposes tool dependence: Claude Opus 5.5 falls from 57% to 4% without tools — ArtificialAnlys · 2026-11-12
- [source] Weakest models gain most from tools: MiMo-V2.6-Pro jumps 5% to 43% on VIALS-AA — ArtificialAnlys · 2026-11-12
- Tool use cost overhead varies wildly: Qwen3.8 Max pays 28× per task, GPT-6 models barely 1.3× — ArtificialAnlys · 2026-11-12
- [source] VIALS-AA leaderboard goes live: compare models on score, cost and tool calls — ArtificialAnlys · 2026-11-12