ScreenBench: AI agents miss hidden wet-lab errors, top score just 0.49
danielmckinn0n · x · 2026-10-30
Gamow Labs released ScreenBench, testing whether AI agents notice hidden errors in wet-lab screening data — the kind of real-world QC that comp-sci people often ignore (was the plate loaded correctly? did the liquid handler actually dispense?).
Benchmark design
- 182 tasks built from real cell-viability screening plates and dosing records, with 1-3 undisclosed errors planted in each
- Tasks ask routine questions about compound activity, identity, or location; answering correctly requires spotting and correcting the errors
- Seven error types: 180° readout rotation, row/column swaps, wrong well order, and more; an "edge effect" serves as a decoy
Results
- Gemini 4 Argon scored 0.49, Opus 5.5 close behind at 0.47, GPT 6.1 Sol just 0.12
- When told which error types to look for, all three agents jumped to 0.94-0.96
- Blind spots differ: on source-exchange errors Gemini gets 94% vs Opus 40%; rotated plates are the reverse (Opus 97%, Gemini 77%); line swaps and well-order errors get past all three
The team hopes labs will hill-climb this task and invites collaboration on reducing error in high-throughput biology.
More from AGI Musings
- PE partner: outsourced dev shops refuse AI yet try to bill clients for it — alex_verem · 2026-10-30
- Are money maximizers safer than paperclip maximizers? A post deemed to settle the debate — eigenhector · 2026-10-30
- Rob LeClerc: We're in the biggest bubble ever, yet stocks crash on unmet demand — robleclerc · 2026-10-30
- Dev: my AI interaction flipped from 95% typing to 95% speaking in 3 weeks — granawkins · 2026-10-30
- Allie Miller: Why Is AI Still So Painfully Single-Player? Multiplayer AI Is Missing — alliekmiller · 2026-10-30
- Stanford prof: think first, then use AI—delegating thinking early erodes your mind — anshulkundaje · 2026-10-30