BioSecBench-Structure: a benchmark for agentic molecular structure reasoning
kenbwork · x · 2026-11-13
Ken Branch introduces BioSecBench-Structure, a structural-reasoning benchmark measuring how well agents interpret molecular structures to assess pathogen targets and support vaccine/antibody/drug development.
Example tasks:
- Count open receptor-binding domains across nine cryo-EM coronavirus spike conformations;
- Decide whether inhibitor screening against a bacterial pilus-assembly protein needs a single template or an ensemble;
- Distinguish two Pseudomonas aeruginosa drug candidates differing by a small chemical modification.
The attached leaderboard shows endpoint pass rates: Fable 5.2 leads at 52.3%, followed by Fable 5.2 (43.7%) and Opus 5.5 (39.7%); DeepSeek V4.1 scores 27.6% and Kimi K3 21.8% — structural reasoning remains hard for frontier models.
More from Safety
- Anthropic roadshow showcasing Accenture as partner raises doubts about auditor independence — DKokotajlo · 2026-11-13
- Harvard FraudBench: AI fraud rate jumps from 3% to 21.4% as agents act on your behalf — rohanpaul_ai · 2026-11-13
- AI SOC monitoring agents emerge as best practice; KV cache optimization called critical to cyber safety — AccBalanced · 2026-11-13
- ROGUE benchmark flagged frontier-agent incorrigibility months before the HuggingFace incident — aran_nayebi · 2026-11-13
- Western University reviews $72 AI tutor after student complaints over cost and privacy — Afinetheorem · 2026-11-13
- Families protest in 7 US cities over Google's inability to offer AI-free Search on school Chromebooks — Ars Technica AI · 2026-11-13