FULL STORY
DeepSeek V4.1-Flash: Benchmark Dispute and Mixed Evaluations
DeepSeek's V4.1-Flash-1017 sparked debate after its claimed 42.3% Terminal-Bench score drew third-party skepticism, while reviewers found it the least hallucinatory in the family but weaker in knowledge and long-context tasks.
2026-10-19 ~ 2026-10-19 · 2 episodes · 5 posts
Episode 1 · DeepSeek V4.1-Flash-1017 Posts Lowest Hallucination Rate but Regresses on Knowledge and Long Context (2026-10-19, 3 posts)
DeepSeek's V4.1-Flash-1017 achieves the lowest hallucination rate among DeepSeek models and ranks seventh on Artificial Analysis' physics reasoning benchmark, but reviewers note regressions in knowledge and long-context ability, with its claimed cost advantages questioned.
- DeepSeek V4.1-Flash-1017 is its least hallucinatory model yet, but regresses on knowledge and LCR — teortaxesTex · 2026-10-19
- DeepSeek V4.1-Flash-1017 Is the Least Hallucinatory DeepSeek Model, but Pareto-Dominated by Sol — teortaxesTex · 2026-10-19
- DeepSeek 1017 Scores 24% on CritPt Physics Benchmark, Cost Savings Questioned — teortaxesTex · 2026-10-19
Episode 2 · DeepSeek V4.1 Flash benchmark score disputed by third-party tests (2026-10-19, 2 posts)
DeepSeek-V4.1-Flash-1017 reports 42.3% on Terminal-Bench 4.0, beating GLM-5.3, but third-party tests show scores over 10 points lower; some users argue the official result is real but relies on DeepSeek's own harness.
- DeepSeek-V4.1-Flash-1017 scores 10+ points lower on third-party Terminal bench tests — DigiDecode_ · 2026-10-19
- DeepSeek V4.1 Flash 1017 reportedly scores 42.3 on Terminal-Bench 4.0, beating GLM-5.3 but only with its own harness — teortaxesTex · 2026-10-19