12-run benchmark: Qwen3.8-Flash-Next trails Opus 5.5 by 14 points at 1/50th the cost
deepu105 · reddit · 2026-10-21
A controlled six-way benchmark of a real Rust feature (saving/restoring llama-server's KV cache) across Claude Code and Pi: each combination run twice for 12 PRs, scored by Opus 5.5, Flash-Next, and Kimi K3 on seven review questions plus CI.
Key results:
- Claude Code + Opus 5.5 (xhigh): 87% avg, 43.5 min, $17.78 API cost
- Claude Code + Opus 5.5 (medium): 78%, $6.52
- Claude Code + Flash-Next (medium): 69%, just $0.32 (ran locally for electricity cost)
- Pi + Flash-Next variants: 63-68%
Takeaways: Opus still leads on code quality (82% vs 68%/66%), with the gap mostly in edge cases, tests, and docs — core functionality was close. Flash-Next took 1.1-2.1x longer at equal effort. xhigh helped Opus modestly at 2.7x cost; for Flash-Next, medium matched xhigh and was faster. CI passed on 4/4 Claude Code PRs but only 1/4 Pi PRs. All 12 PRs are public.
Related event: Benchmarks Show Qwen3.8-Flash-Next Near Opus 5.5 at 1/50 the Cost(3 posts)→
More from coding & agent
- Ragged Blocks: a new way to render nested structure within typographic layout — neurocy · 2026-10-21
- Twenty, the open-source AI-native CRM alternative to Salesforce, nears 59k stars — twentyhq · 2026-10-21
- freehire: open-source AI job search engine hits 2k GitHub stars — strelov1 · 2026-10-21
- Running multiple Claude instances to talk to each other slows everything down — rickasaurus · 2026-10-21
- Vibe coding without judgment is just 'lottery with syntax,' developer argues — robertonappe · 2026-10-21
- Indie dev vibe-codes a beehiiv alternative, hits $7,836 ARR in one week — marclou · 2026-10-21