DeepSeek V4.1 Flash tops Chinese models in coding blind test, but drops 17 points when switching clients
teortaxesTex · x · 2026-09-09
A 7-way "model + coding client" blind test on the same production-grade code task put DeepSeek V4.1 Flash + Claude Code first among Chinese models at 76.69, ahead of Qwen 3.8 Flash (75.13), Kimi K3-256K (70.73), Qwen 3.8 Max (69.58) and GLM 5.3 (64.72).
- Strengths: 88 on security/privacy, highest scores on failure/concurrency safety and performance, 99%+ cache hit rate, finished in 49 minutes — faster than larger models.
- Weaknesses: functional correctness 78 lost to Qwen 3.8 Flash's 82; failure-safety absolute score only 54; not first in test quality, architecture, compatibility or delivery.
- The host client is the biggest variable: the same model dropped 16.77 points (to 59.92) when run under DSH, showing how much tool integration and execution flow matter.
Takeaway: Flash already competes with flagships, but engineering edge cases remain — and the client itself is currently the biggest variable. A single-task engineering eval, not a general capability ranking.
More from coding & agent
- Claude Code Drives 46.5% of Coding-Agent Traffic to Hugging Face, Codex Second — vanstriendaniel · 2026-09-09
- AI Engineering Buildcamp Reveals Six Hands-On Projects, from RAG to Multi-Agent Systems — Al_Grigor · 2026-09-09
- What's the most common way people actually set up multi-agent coding workflows? — Necessary-Apple337 · 2026-09-09
- New CLI tool scrapes WeChat articles into Markdown from any post URL — vista8 · 2026-09-09
- The expensive part of coding agents isn't the agents—it's the silent retries ($900 for one task) — mrtrly · 2026-09-09
- A green test suite proves an agent can write tests, not that the feature works — Future_AGI · 2026-09-09