105 planted bugs tested: local Qwen3.8-27B nearly matches Claude Opus at bug fixing
PawelHuryn · x · 2026-09-17
The author benchmarked local LLMs on real work: two repos with 105 planted bugs to find and fix. Results: Opus 4.8 (max) fixed 15, Qwen3.8-27B 8-bit fixed 10.7, Sonnet 5 (high) 9, Gemma 4 31B only 4, and gpt-oss-120b scored 0 (useless).
The surprise was the 8-bit Qwen3.8-27B: at 28.6 GB, it runs comfortably on a 64 GB Mac mini with a 200K context window. Not as smart as frontier closed models, but capable of real, useful bug-fixing work at the cost of electricity alone.
Related event: Bug Hunt Bench: local Qwen3.8-27B nears Claude Opus in bug fixing(3 posts)→
More from Models
- Claude upgrade-page traffic rebounded to 17.6M in September, near May's 18.6M peak — firstadopter · 2026-11-02
- Open-source GLiNER-PII scrubs 13 identifiers from a discharge note in 550ms on GPU — MaziyarPanahi · 2026-11-02
- OpenAI Dots user says bloat and broken infra drove him to try newcomer Muse — Koala_Confused · 2026-11-02
- Fable 5.5 reportedly one-shots animation in pure code as Anthropic's coding bet pays off — Hesamation · 2026-11-02
- One agent goal has been running for 4 days, burning 30K tokens — CtrlAltDwayne · 2026-11-02
- Daily AI Brief: Gemini 4 Carbon spotted on Arena, DeepSeek open-sources coding agent — testingcatalog · 2026-11-02