DeepSeek V4.1 Flash tops Chinese models in coding blind test, but drops 17 points when switching clients
teortaxesTex · x · 2026-09-09
A 7-way "model + coding client" blind test on the same production-grade code task put DeepSeek V4.1 Flash + Claude Code first among Chinese models at 76.69, ahead of Qwen 3.8 Flash (75.13), Kimi K3-256K (70.73), Qwen 3.8 Max (69.58) and GLM 5.3 (64.72).
- Strengths: 88 on security/privacy, highest scores on failure/concurrency safety and performance, 99%+ cache hit rate, finished in 49 minutes — faster than larger models.
- Weaknesses: functional correctness 78 lost to Qwen 3.8 Flash's 82; failure-safety absolute score only 54; not first in test quality, architecture, compatibility or delivery.
- The host client is the biggest variable: the same model dropped 16.77 points (to 59.92) when run under DSH, showing how much tool integration and execution flow matter.
Takeaway: Flash already competes with flagships, but engineering edge cases remain — and the client itself is currently the biggest variable. A single-task engineering eval, not a general capability ranking.
More from coding & agent
- SF 'Code Engineering' event picks up where 1,000-person harness engineering meetup left off — LangChain · 2026-10-27
- Prompt turns Claude Code into an iPhone backup forensics tool exposing 30-day 'deleted' photo copies — alex_verem · 2026-10-27
- Prompt turns Claude Code into a forensics tool finding deleted iPhone photos in backups — alex_verem · 2026-10-27
- Phones won't be mobile IDEs but control planes for agent fleets, argues Yuchen Jin — Yuchenj_UW · 2026-10-27
- FrontierCS 2 launches on Kaggle: 48 tasks make agents beat published CS papers — MeganRisdal · 2026-10-27
- Agent Field Kit launches $29 security runbooks for production AI agents — alvelda · 2026-10-27