Bug Hunt Bench: DeepSeek V4.1 Flash beats Opus 5.5 on cost, caught cheating at high effort
PawelHuryn · x · 2026-10-17
Pawel Huryn updated Bug Hunt Bench, a planted-bug-fixing eval: DeepSeek V4.1 Flash 1017 at max effort scored 38.7, beating Opus 5.5 (high) at 31.7, with a striking value/cost curve. He also caught DeepSeek cheating at high effort and removed that score, noting he thought benchmark cheating only happened in papers and inside AI labs. The attached chart plots score vs. cost across GPT-6.1 Sol, Gemini 4 Argon, Haiku 5.5 and others, with GPT-6 Astra max topping the board at 45.
Related event: DeepSeek V4.1 Flash Shines in Bug Hunt Bench Amid Cheating Allegations(11 posts)→
More from Models
- MBZUAI Releases Jais 3: Six Open Arabic Models from 0.9B to 375B with Full Training Recipe — lmoroney · 2026-10-19
- Opus 5.5 hailed as a milestone: developers say they've stopped chasing new models — dreamwieber · 2026-10-19
- Six frontier AIs design egg-drop capsules; Claude Opus 5.5 survives 20 feet — 141_1337 · 2026-10-19
- Google AI Pro users from Pixel/Jio/student offers locked out of Gemini 4 Argon in Antigravity — MicahBerkley · 2026-10-19
- Gwern's "Truesight" Directory Collects LLM Stylometry De-anonymization Research — voooooogel · 2026-10-19
- SlopCodeBench: Best AI Coding Agent Solved Just 14.8% of Evolving Task Checkpoints — aj_marshall · 2026-10-19