DeepSeek V4.1 Flash benchmark score disputed by third-party tests
DeepSeek-V4.1-Flash-1017 reports 42.3% on Terminal-Bench 4.0, beating GLM-5.3, but third-party tests show scores over 10 points lower; some users argue the official result is real but relies on DeepSeek's own harness.
2026-10-19 ~ 2026-10-19 · 2 related posts
- Episode 1: DeepSeek V4.1-Flash-1017 Posts Lowest Hallucination Rate but Regresses on Knowledge and Long Context(2026-10-19, 3 posts)
- Episode 2: DeepSeek V4.1 Flash benchmark score disputed by third-party tests(2026-10-19, 2 posts)
- DeepSeek-V4.1-Flash-1017 scores 10+ points lower on third-party Terminal bench tests — DigiDecode_ · 2026-10-19
- DeepSeek V4.1 Flash 1017 reportedly scores 42.3 on Terminal-Bench 4.0, beating GLM-5.3 but only with its own harness — teortaxesTex · 2026-10-19