New VIALS-AA leaderboard: Gemini 4 leads at 64%, Claude costs 29x more per task
ArtificialAnlys · x · 2026-11-07
Artificial Analysis launched VIALS-AA, a leaderboard for Handshake AI's VIALS benchmark: 161 tasks across eight life-sciences domains requiring models to interpret gel blots, flow cytometry plots, plasmid maps, and phylogenies. Each task runs 3x in the open-source Stirrup agent harness, graded by GPT-6 Luna (medium), reporting pass^3.
Key findings:
- Top three within 3 points: Gemini 4 Carbon (high) leads at 64%, GPT-6 Astra (max) 63%, Claude Opus 5.5 (max with fallback) 61%
- Cost gap: Opus 5.5 averages $1.64/task, 29x Sol ($0.056), driven by turn counts (25–27 for Claude vs 8 for GPT-6 models)
- Open weights trail by 19 points: Kimi K3.1 Pro leads at 45%, ahead of MiMo-V2.6-Pro at 43%
- Safety refusals on 14 of 161 tasks; Claude Sonnet 5.5 refused the most (6)
More from Models
- Meta ships Muse Wonder specialized models: 2.4x less forgetting, 67% better unsticking — AIatMeta · 2026-11-07
- Generated worlds tested at runtime: local interactions 83/100, stateful workflows just 30/100 — nptacek · 2026-11-07
- Steering a single neuron gives Qwen3-30B a new identity in ~1-2% of replies — Sauers_ · 2026-11-07
- OpenAI and Anthropic stuck in a release standoff as both hold back their next models — kimmonismus · 2026-11-07
- Deep Learning Weekly #480 ships with 'Gemini 4 Carbon' mention and model-routing paper — dl_weekly · 2026-11-07
- PostTrainBench: AI agents' post-training score jumps from 23% to 45% in 7 months — karinanguyen · 2026-11-07