Planted 105 bugs in 2 repos: Qwen3.8-27B 8-bit run 3x beats a single Opus 4.8 run
PawelHuryn · x · 2026-09-17
PawelHuryn tested local LLMs on real work: 2 repos with 105 planted bugs, asking models to find and fix what they can. Single-run scores: Opus 4.8 (max) found 15, Qwen3.8-27B 8-bit 10.7, Sonnet 5 (high) 9, Gemma 4 31B only 4, and gpt-oss-120b 0 (useless).
The surprise was Qwen3.8-27B: the 8-bit model is about 28.6 GB and comfortably runs on a 64 GB Mac mini with a 200K context window. Running it three times beats a single maxed-out Opus 4.8 run — a strong data point for local models on real engineering tasks.
Related event: Bug Hunt Bench: local Qwen3.8-27B nears Claude Opus in bug fixing(3 posts)→
More from coding & agent
- awa-video-reel: free Claude Code skill auto-edits shorts and records website B-roll — TawohAwa · 2026-11-02
- Econ PhD Job Market Database Hits 900+ Candidates, Open to Codex and Claude Code — aniketapanjwani · 2026-11-02
- Cron works by hand, fails on schedule: a 7-point debugging checklist — petrusenko_max · 2026-11-02
- Local quants vs Claude on 9 real private bugs: 3-bit costs thinking time, not score — FeydRowan · 2026-11-02
- Build products for AI, not humans: MCP lets Claude Code run everything — gefei55 · 2026-11-02
- Solo dev launches hosted MCP server with 31 tools scheduling to 20+ social networks — troyizzle · 2026-11-02