DeepSeek-V4.1-Flash-1017 scores 10+ points lower on third-party Terminal bench tests
DigiDecode_ · reddit · 2026-10-19
A Redditor compared DeepSeek-V4.1-Flash-1017's Terminal bench 4.0 results across sources: the model card reports 42.3%, while Artificial Analysis reports 34% and the Vals index reports just 31.82% — a gap of over 10 points versus the official figure, raising questions about benchmark methodology or model card claims.
More from Models
- Retrained "bee model" drifts into dreamy, detached trace-like outputs — Sauers_ · 2026-10-19
- One prompt in a normal ChatGPT chat yields an 87-second chalkboard explainer video — mallow610 · 2026-10-19
- Claude Opus normal chats burn code quota, user says ChatGPT sub is far more generous — mallow610 · 2026-10-19
- A ~56KB meditative HTML page one-shot with Gemini using Antigravity — Saboo_Shubham_ · 2026-10-19
- OpenAI models top Artificial Analysis token burn at 197k output tokens per task — teortaxesTex · 2026-10-19
- Redditor begs OpenAI for a work-only ChatGPT plan, says Anthropic's allowances beat it for agents — bofors14 · 2026-10-19