Qwen 3.8 27B scores 72.9 on Aider benchmark, tying Gemini 2.5 Pro
Baldur-Norddahl · reddit · 2026-08-24
The author ran the Aider benchmark with Qwen 3.8 27B FP8 + FP8 KV cache at 256K context on vLLM. Score: 72.9 — matching Gemini 2.5 Pro (2025-04-12, 72.9), beating Claude Opus 4 (2025-05-25, 72.0) and DeepSeek R1 (71.4).
He notes it's an older benchmark, but it's remarkable that a MacBook now matches SOTA models from just over a year ago. Real-world performance is even better since the harness improved: with the DeepSeek Harness it would solve most if not all Aider tests, though often using more than 2 turns.
More from Models
- Google says teams across the company are using Gemini 4 Argon on its hardest engineering problems — CurieuxExplorer · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Leak: Moonshot's Kimi flagship 'x4' done with pre-training, K3.1 in internal testing — lyraxana · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15
- NVIDIA ships two open Nemotron 3.5 Bolt models: a 74B-A9B MoE and a mobile-friendly E4B — HankYeomans · 2026-10-15