Study Finds Frontier AI Fixes Only 26% of Security Vulnerabilities

A joint study by Off-by-1 Labs and 1Password tested over 6,000 AI-generated patches from Claude and ChatGPT on real vulnerabilities, finding only about 26% actually fixed the bugs, suggesting unreviewed AI patches can be net-negative.

2026-09-06 ~ 2026-09-06 · 2 related posts

• Frontier AI models fix only 1 in 4 security vulnerabilities correctly, report finds — Evgenii42 · 2026-09-06
• 1Password study: AI fixes only 26% of security bugs, unreviewed LLM patches are net-negative — GaryMarcus · 2026-09-06