BioSecBench reveals AI agents struggle to infer pathogen properties, top score under 51%
kenbwork · x · 2026-08-28
Researchers introduced BioSecBench-Function, a verifiable benchmark testing whether AI agents can infer functional properties of viruses, bacteria, and toxins from data.
- Scope: Contains 111 deterministic evaluations covering five threat axes: transmissibility, immune escape, virulence/toxicity, drug resistance, and fitness.
- Results: The strongest configuration (Opus 5 with Claude Code) achieved only a 50.4% endpoint pass rate. Grok 4.6 led in overall pass rate at 44% when refusals are counted as failures.
- Methodology: Agents receive data from studies (e.g., deep mutational scanning, X-ray crystallography) and must produce structured answers graded against ground truth.
Related event: BioSecBench: AI Agents Fall Short on Pathogen Function Inference(3 posts)→
More from Safety
- CognitionX Dubai demo walks through full Microsoft Entra ID kill chain, from discovery to Global Admin — lee_stott · 2026-10-15
- DiPLab report: Scale AI's Outlier runs 16 trackers harvesting biometric data from EU workers — AntonioCasilli · 2026-10-15
- Your breathing gives you away: Wi-Fi signals can reveal your breath rhythm — TinfoilTricorn · 2026-10-15
- Cohere acquires Aisle, whose defensive cyber tech found 200+ High/Critical CVEs in curl, FFmpeg and more — aidangomez · 2026-10-15
- Podcast: AI is making pentesting more profitable and bug bounty more competitive — rez0__ · 2026-10-15
- Chatbots' bad health advice leaves no trace, warn researchers in Nature Health paper — alex_verem · 2026-10-15