RespSurvBench: AI scores just 25.5% on respiratory outbreak surveillance across 110 evals
kenbwork · x · 2026-11-06
The team behind RespSurvBench introduces a benchmark of 110 evaluations spanning diagnostic assays, pathogen discovery, viral evolution and transmission, and population surveillance. Across 21 models and 34 model–harness configurations, the highest pass rate was 25.5% (Claude Opus 5.5 in Pi), with refusals and execution errors counted as failures. Common scientific mistakes include treating missing observations as negatives, mismatching records, and confusing measurement artifacts with true biological signal. The team argues AI will become core infrastructure for respiratory outbreak detection.
More from Research
- AI reads 2,000 historical illustrations: individual errors, near-perfect aggregate patterns — _akpiper · 2026-11-23
- ReLU networks have zero second derivative almost everywhere — why PINNs need smooth activations — bravo_abad · 2026-11-22
- Sinclair lab uses AI to find chemical age-reversal molecules, mouse tissues rejuvenated in 4 weeks — thederbiedone · 2026-11-22
- Aaronson on reported LLM proof of the Riemann Hypothesis: 17.4M lines of Lean code — jedisct1 · 2026-11-22
- Tufazip losslessly compresses 1GB Wikipedia to 105MB, best neural-network-only entry on LTCB — RichmanRonald · 2026-11-22
- Fine-tuned 2B models beat OpenAI o3 and Gemini 2.5 few-shot on molecular classification — madeofAjala · 2026-11-22