Science Prediction Universe benchmark: best agents score just 0.486 at predicting wet-lab outcomes
liangchen5518 · x · 2026-11-16
Researchers unveiled Science Prediction Universe (SPU), a benchmark testing whether AI agents can predict wet-lab experiment outcomes before they're run:
- Tasks: 100 public verified tasks across chemistry, materials science, biology and medicine, plus 16 private tasks from unpublished wet-lab experiments. Given an experimental setup and candidate conditions, agents must rank candidates by measured outcomes.
- Coverage: 13 agent configurations spanning 12 frontier models across two harnesses (Codex and Claude Code).
- Findings: Performance is far from saturation with large model gaps. On the public leaderboard, Claude Opus 5.5 leads at 0.486 mean score, followed by Claude Fable 5.1 (0.438), Claude Opus 5 (0.420) and GPT-6 Astra medium (0.416); Chinese models trail, including Doubao Seed Evolving (0.315), Kimi K3.1 Pro (0.284), DeepSeek V4.1 Flash (0.264) and GLM-5.3 Flash (0.237).
More from Models
- NVIDIA's Free Nemotron 3.5 Bolt E4B: A 4GB Model That Runs Offline on Almost Any Laptop — alex_verem · 2026-11-16
- Gemini 4 Hydrogen doc analysis tested: one prompt yields report plus comparison table — technextpreneur · 2026-11-16
- Anthropic S-1: Models Older Than 6 Months Earn Under 20% of Revenue — every · 2026-11-16
- OpenAI weekly: ChatGPT Wallet rollout, Odessia acquisition, voice modes retiring Dec 11 — btibor91 · 2026-11-16
- AI-built 367K-parameter LM fits on a floppy disk and runs on a Pentium — RichmanRonald · 2026-11-16
- Inference providers are not hosting Perplexity's decider model despite production demand — bo_wangbo · 2026-11-16