VulnPR-100: DeepSeek V4.1 Flash finds 54 vulns for $7.53, crushing rivals on cost
teortaxesTex · x · 2026-10-24
VulnPR-100 benchmark: Kimi K3.1 Pro vulnerability hunting tested
BugBunnyai ran Kimi K3.1 Pro through the VulnPR-100 benchmark: 100 PRs with known vulnerabilities, one hour per PR to find the bugs.
- K3.1 Pro found 47/100 vulnerabilities for $124.69 — 10 more than Kimi K3 while costing $20.54 less; cost per finding dropped from $3.93 to $2.65 (33% improvement)
- It beat GPT-6 Astra and Muse Wonder 1.0 (both at 46) and trailed Claude Opus 5 by just one finding at 3.6× lower cost
- DeepSeek V4.1 Flash 1017 still leads: 54 vulnerabilities for only $7.53 — seven more findings at roughly 16.6× lower cost
Verdict: a strong generational improvement for Kimi, but DeepSeek remains the model to beat on price-performance.
More from Models
- Abacus AI to launch SuperComputer: text prompt to custom-trained LLM in an hour — bindureddy · 2026-10-24
- One-prompt Minecraft build of an RTX 5090 sparks model-vs-Opus 4.5 poll — adonis_singh · 2026-10-24
- Abliterated DeepSeek running on a Pi is the AI power-user flex of the day — EXM7777 · 2026-10-24
- Tesla engineer: Cybercab camera tuning done with Grok, coding fixed-function hardware directly — yunta_tsai · 2026-10-24
- Kimi K3.1 Pro costs $1.03 median per task but lands just off Agent Arena's Pareto frontier — arena · 2026-10-24
- Kimi K3.1 Pro vs Claude Opus 5.5 benchmark comparison released — alejandroll10 · 2026-10-24