DeepSeek V4.1 Flash scores 24/105 for $1.80 in community eval test
PawelHuryn · x · 2026-09-10
Blogger PawelHuryn benchmarked new DeepSeek models on his eval suite: DeepSeek-V4.1-Flash at max effort scored 24/105 for $1.80 in 42.6 min (19/105 at $0.31 on high effort). Comparisons: Gemini 3.8 Flash (high) 20/105 at $9.78; GLM-5.3 19/105 at $19.73 in 66.7 min. A follow-up test of DeepSeek V4 Pro (max) scored 16/105 at $1.89 vs V4.1 Flash (max) 24/105 at $1.08—significant gains at far lower cost. Updates tracked on GitHub.
Related event: Bug Hunt Bench Ranks Frontier Coding Models on 105 Real Bugs(13 posts)→
More from Models
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- ARC Prize redesigns leaderboard: highest reasoning effort by default, CSV export — mhmazur · 2026-10-27
- Cohere: open-weight Qwen3.8-Flash-Next tried uploading probe file to Beijing cloud bucket in sandbox — wstone_bd · 2026-10-27
- Model refuses wrong psychiatric advice in deliberately misleading test — FellMentKE · 2026-10-27
- Clinical System One v0.2: top clinical AI drops from 97% to 65% on harder cases — MaziyarPanahi · 2026-10-27
- ChatGPT keeps telling people 404 Media is tech support for big corporations — 404 Media · 2026-10-26