Independent Replication Matches DeepSeek V4 Flash 82.7% on Terminal-Bench
Exciting-Camera3226 · reddit · 2026-08-09
A third-party evaluator successfully independently replicated the Terminal-Bench 2.1 score of DeepSeek V4 Flash 0731 using the public Ante 0.preview.71 framework.
Configured with 89 tasks and 5 trials per task (445 total trials), the model was run via OpenRouter with max reasoning effort and no skills enabled. It achieved 368 successful trials, yielding an 82.7% accuracy (±1.79 SE).
This result exactly matches the score previously reported by DeepSeek using an unreleased "DeepSeek Harness minimal mode," proving the model's sensitivity to the testing harness and providing credible public evaluation data for the community.
More from Models
- willcb: Opus 5.5 doesn't smell bigger, it smells like they fixed the data — willcb · 2026-10-16
- Unverified claim: Apple and Meta's 816B-parameter Muse Spark Open runs on two Mac Studios — TansuYegen · 2026-10-16
- Gemini 4 model card: model eliminated months of trial-and-error on bioweapons research — Puzzleheaded-King584 · 2026-10-16
- Every AI picks 7: Anthropic engineer's rule for escaping model taste convergence — rubenhassid · 2026-10-16
- Brain teaser: strip all attention, MLP and embedding matrices from a Llama 3 — how many trainable params remain? — Sauers_ · 2026-10-16
- Lean v4.35.0 ships verified kernels — one written and proven consistent by Claude — geoffreyirving · 2026-10-16