VIALS benchmark: frontier models flunk 161 life-sciences image interpretation tasks
niloofar_mire · x · 2026-11-07
- Handshake AI released VIALS (arXiv:2608.21357), a visual QA benchmark of 161 interpretation tasks across 8 domains, asking models to read real biotech artifacts — gel blots, flow cytometry plots, plasmid maps, phylogenies — and draw specific scientific conclusions, using industry-workflow images rather than polished textbook figures.
- Artificial Analysis launched the companion VIALS-AA leaderboard, running each task three times with Stirrup, its open-source agent harness with code execution and image inspection tools.
- Key finding: frontier models struggle badly, lacking both scientific knowledge and visual perception; PhD-trained scientists handle these tasks in minutes. Until AI closes that gap, its usefulness in industrial life-sciences workflows stays limited.
- Why it matters: visual interpretation of assays drives critical decisions in drug discovery — a core bottleneck for AI in biopharma.
More from Models
- Kimi K3.1 Pro turns a photo into a 3D Three.js scene, with open-sourced template — demian_ai · 2026-11-07
- Nous Research releases Arete-27B, cutting a Chinese open model's propaganda rate from 22% to 1% — NousResearch · 2026-11-07
- Claim: GPT-6 Astra's robotics skills aren't emergent — it trained on robotics-specific data — gleech · 2026-11-07
- LMArena launches Code Arena with WebDev categories for head-to-head coding tests — arena · 2026-11-07
- Empero's Manta claims a 100-million-token context window in a single prompt — emdeka87 · 2026-11-07
- Team bans commas and noun-phrase clustering to scrub Claude's AI writing tells — layer07_yuxi · 2026-11-07