Tool use nearly doubles VIALS-AA scores: tool-less Claude Fable 5.2 loses to Claude Haiku 5.5 with tools
ArtificialAnlys · x · 2026-11-12
Artificial Analysis reran its VIALS-AA benchmark (visual reasoning in life sciences, 161 tasks) in a one-shot format without tools, comparing against the default agentic setup where models think, run code and view images in a sandbox.
- Across models, scores nearly double with tools: 25% to 47% on average
- Claude Fable 5.2 (max) leads at 67% with tools ($4.20/task); without tools it drops to 44% ($0.73)
- Claude Haiku 5.5 (max) hits 49% with tools at just $0.07/task — beating tool-less Fable at roughly a tenth of the cost
Takeaway: smaller models in an agent harness can beat single-shot large models on both cost and performance for scientific image interpretation.
More from Models
- Fable 5.2 cuts refusals dramatically but slides to 40th & 53rd on BullshitBench — burny_tech · 2026-11-12
- Artificial Analysis: Upgrading Claude model tier beats maxing reasoning effort on cost — ArtificialAnlys · 2026-11-12
- Google Showcases Gemini 4 Carbon: Building a Custom CUDA/C++ VFX Engine From Scratch — burny_tech · 2026-11-12
- Claude iOS spotted adding 'Saved logins' for signing into websites, fueling Duet speculation — testingcatalog · 2026-11-12
- Fable 5.2 reverse-engineers how 1400s composers wrote four-voice counterpoint, then composes — imjustnewatai · 2026-11-12
- GPT-Live talk at VapiCon: OpenAI pushes natural speech with frontier intelligence — juberti · 2026-11-12