Open-Source AI Safety Classifiers Fail to Reliably Block Bioweapon Generation
kenbwork · x · 2026-08-07
Researchers tested 6 open-source AI safety classifiers on 73 biosecurity tasks, including dual-use constructs and routine research. None could reliably distinguish dangerous work from legitimate science.
The best-performing Llama Guard 4 caught 76% of red-team threats but blocked 55% of legitimate research. Other classifiers performed near or below a coin flip. Mistral's newly released Shieldstral, for instance, accepts all requests at its highest accuracy threshold (0% threat catch). Even at its most aggressive setting, it catches only 24% of threats while blocking over half of legitimate research.
Furthermore, the guards disproportionately block routine work mentioning recognizable hazards like Ebola and malaria, and even correct refusals often happen for the wrong reasons.
Related event: Open-Source AI Safety Classifiers Fail to Reliably Block Bioweapons(2 posts)→
More from Models
- Opus 5.5 turns heads with surprisingly polished vertical video demo — FinanceYF5 · 2026-10-16
- Astra 6.1 may arrive before end of month, reads dev's 'we are cooking' tease — haider1 · 2026-10-16
- Open-source OCR model indra-ocr trends on Hugging Face with CPU support — datalab-to · 2026-10-16
- Xiaohongshu's Vela 35B turns anonymized user behavior into supervision, gains +11.5 on VelaBench — AdinaYakup · 2026-10-16
- KOL: No better model than Anthropic's Opus right now, its magic is back — kimmonismus · 2026-10-16
- Users blast Gemini in Chrome's 'draconian' usage caps: memory costs 2-4% per use — North-American · 2026-10-16