Open-Source Safety Classifiers Block Over Half of Legitimate Biological Research
kenbwork · x · 2026-08-07
An empirical evaluation of open-source AI safety classifiers reveals that current models struggle to distinguish between dangerous biological research and legitimate science. Tested across 73 biology tasks, the best-performing model, Llama Guard 4, catches 76% of red-team threats but also falsely blocks 55% of legitimate research requests, highlighting significant over-refusal issues in safety guardrails.
Related event: Open-Source AI Safety Classifiers Fail to Reliably Block Bioweapons(2 posts)→
More from Models
- Artificial Analysis launches Decision Index for the emerging decision model category — ArtificialAnlys · 2026-10-16
- d1-ultra runs 93 ms per task at $32 per 1M tasks, on par with rivals — ArtificialAnlys · 2026-10-16
- Liquid AI's d1-ultra tops Artificial Analysis Decision Index with 85% non-hallucination rate — ArtificialAnlys · 2026-10-16
- Meta open-sources Muse Spark weights, Zuckerberg thanks Apple on X — AIFlow_ML · 2026-10-16
- User claims ChatGPT Pro's $100, $200 and $500 plans share nearly identical usage caps — These-Breadfruit4067 · 2026-10-16
- Users report Gemini's memories can't actually be deleted even after wiping chat history — ThrowRA_Purpose2025 · 2026-10-16