LSCBench: top AI coding agent passes 91.3% of tests but only 25.1% ship securely
krishnan · x · 2026-10-18
LSCBench, an arXiv benchmark (Oct 14, 2026) built by researchers from the National University of Singapore and Alibaba Group (including Qwen team members), tested six frontier coding agents on 450 tasks drawn from 150 real-world vulnerabilities. The headline finding: the best agent passed 91.3% of functional tests, but scored only 25.1% on the combined "works and is secure" measure.
- Task specs only asked for working functionality, implicitly probing whether agents avoid real-world vulnerability patterns on their own.
- The gap lands as AI agents produce 30% more code without a matching increase in shipped software, prompting enterprise security teams to rethink how much autonomy these tools should get.
The poster adds that human code he reviewed at both a big company and startups fared no better on security than AI output.
More from coding & agent
- Marketer shares the one-prompt AI content system behind millions of monthly impressions — VibeMarketer_ · 2026-10-18
- AI influencer builder seeks help: local stack works but hands break and skin looks plastic — Top-Sheepherder-4131 · 2026-10-18
- College football is the ideal backdrop for developing with agents, says this dev — thechrisperry · 2026-10-18
- DeepSeek V4.1 Flash as a coding agent: fast at the core, but prone to rabbit-holing on polish — teortaxesTex · 2026-10-18
- Agent checkout needs password manager support: dev wants 1Password in Grok's payment dialogs — lucasmeijer · 2026-10-18
- OpenAI's Codex weekly usage reset rolls out, limits now shown as 100% left — ChrisUniverse · 2026-10-18