OSS Scanner's first scans report 77 critical and 803 high findings; LLMs now find 85% of CyberGym bugs
ClaudeDevs · x · 2026-10-20
Anthropic shared early results from OSS Scanner: 77 critical and 803 high model-rated findings reported to maintainers, with scanning ongoing.
- On the CyberGym benchmark, LLMs went from finding under 20% of vulnerabilities at the start of last year to over 85% this year
- Humans validated 6,000 of 29,000 candidate vulnerabilities; maintainers increasingly ask for bulk unverified reports with patches — nearly 5,000 sent
- With exploits writable in minutes, faster discovery is now a security race
More from Safety
- Researchers pull real leaked API keys from LLMs using only API output — nordicinst · 2026-10-20
- Microsoft says AI agents should be treated like insider threats as 'frontier models can't be trusted' — TechCrunch AI · 2026-10-20
- Studios Want AI to Mint New Franchises, but Pure AI Content Can't Be Copyrighted — WhatTheLJW · 2026-10-20
- French AI-generated novel axed from Goncourt longlist; writers demand AI text watermarking — zetalyrae · 2026-10-20
- NYC to take 15% of QFab IP proceeds and three board seats in landmark startup deal — MJBiercuk · 2026-10-20
- Fake Anthropic employee accounts run a novel X follow-based phishing scheme — edgarpavlovsky · 2026-10-20