Gemini 3.8 Flash ties for top of DeepSWE at 74%, but burns 1.34x more output tokens
haider1 · x · 2026-09-03
- Top of DeepSWE: Gemini 3.8 Flash now ties at the top of DeepSWE, a benchmark for long-horizon software engineering tasks, scoring 74%, up from Gemini 3.7 Flash's 65%.
- Token efficiency takes a hit: it outputs about 143k tokens per task vs 107k for 3.7 Flash — 1.34x more.
- Cross-model comparison: the author notes it scores 59 on the AA Intelligence Index, a big jump, yet uses more output tokens per task than the GPT-5.6 family, Opus 5, Fable 5, and even Fable 5.1 — intelligence improved, but token efficiency looks rough.
More from Models
- Mocked benchmark shows DeepSeek's document chart grounding badly misaligned — burkov · 2026-10-18
- Dev complains Claude's habit of saying "not done" is his biggest pain point — jasonkneen · 2026-10-18
- Open-weight TYGER speech decision model makes 80ms turn-taking calls for full-duplex voice agents — solyarisoftware · 2026-10-18
- Paying Google AI subscriber blocked: custom-domain Workspace accounts can't access AI Studio — ChrisRackauckas · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- Users press OpenAI on long-delayed Adult Mode a year after it was promised — ___Patrice___ · 2026-10-18