1,665 model runs across 27 repos: open-source models beat closed ones at security audit, dev claims
Fluffy-Ad-889 · reddit · 2026-09-08
A developer claims to have run local and cloud models against 27 public GitHub codebases for security auditing — 1,665 model runs and 1,067 findings over two weeks, with a verifiable results database published. Spot-check accuracy: minimax-m3 10/12, deepseek-v4-flash 6/6, glm-5.1 5/5, gpt-oss-20b 5/5, while claude-opus-5 scored 0/8.
The takeaway: open-source models dominate cybersecurity use cases, and the author offers to share queries and methodology for reproduction. Caution: the claim that HuggingFace was "attacked by OpenAI" and defended itself with GLM 5.2 has no reliable source, and some model names are dubious — the data is unverified.
More from Models
- Astra is 'actually quite good' at using a browser, though still not fast — jobergum · 2026-09-08
- Codex tip: Astra now reads your remaining usage % so you can budget in plain English — Dimillian · 2026-09-08
- Matt Shumer calls GPT-6 Pro "a monster" in unverified hype tweet — msg · 2026-09-08
- "The old models just sucked": stronger models redefine how intensely you use AI — teortaxesTex · 2026-09-08
- Astra announces global usage reset for all paid subscriptions today — infoxiao · 2026-09-08
- New Codex 7-day limit visualization shows red when you're running ahead of your quota — lucasmeijer · 2026-09-08