105 planted bugs tested: Kimi K3.1 Pro scores 36, loses on cost-effectiveness
PawelHuryn · x · 2026-10-24
Testing Kimi K3.1 Pro on 105 planted bugs across two real repos: DeepSeek V4.1 Flash 1017 (max) led at 42.7, Muse Wonder Full 1.0 hit 42, Opus 5.5 scored 41.7, while Kimi K3.1 Pro managed only 36 and Muse Spark 1.3 got 32.2. The author's verdict: factoring in cost, 'I see no reason to use this model.' The accompanying Bug Hunt Bench leaderboard shows Sonnet 5.5 Max leading at 51.3, with DeepSeek runs costing as little as $1 versus $154 for Sonnet 5.5.
More from Models
- Block launches Dessa, an AI voice model small enough to run on a MacBook — Polymarket · 2026-10-24
- Claude Max user burns through 99% of weekly limits before Monday reset — iamsahaj_xyz · 2026-10-24
- Google says Gemini 4 Argon found fusion counterexamples and designed 5 lab-verified enzymes in hours — Eric520CC · 2026-10-24
- Claude's redesign draws flak: beloved compact design system now "just looks like Codex" — jrmyphlmn · 2026-10-24
- 180 code-generated paintings compared: Kimi K3.1 Pro vs Claude Opus 5.5 vs Kimi K3 — xeophon · 2026-10-24
- Kimi k3.1 pro aces the pelican-bike test in 45 minutes; author calls benchmark saturated — ChrisGPT · 2026-10-24