RespSurvBench: AI scores just 25.5% on respiratory outbreak surveillance across 110 evals

kenbwork · x · 2026-11-06

The team behind RespSurvBench introduces a benchmark of 110 evaluations spanning diagnostic assays, pathogen discovery, viral evolution and transmission, and population surveillance. Across 21 models and 34 model–harness configurations, the highest pass rate was 25.5% (Claude Opus 5.5 in Pi), with refusals and execution errors counted as failures. Common scientific mistakes include treating missing observations as negatives, mismatching records, and confusing measurement artifacts with true biological signal. The team argues AI will become core infrastructure for respiratory outbreak detection.

Related event: RespSurvBench: Top AI Model Passes Only 25.5% on Respiratory Outbreak Monitoring(4 posts)→

Original post →

More from Research

Research channel →