DeepSeek-V4 Pro Beats Rivals in Cost-Effective Benchmark

DeepSeek-V4 Pro (0813) has demonstrated impressive performance in the DeepSWE benchmark, achieving an accuracy of 88.5% (pass@4) and outperforming GPT-5.6 Sol and Fable 5. With a cost of only $0.24 per task, it shows significant cost advantages. Real-world tests confirm its lower costs compared to competitors like Fable. The model's complementarity with rivals has sparked discussions about routing strategies in the community.

Confirmed

• Main Results: DeepSeek-V4 Pro (0813) achieved 88.5% pass@4 on DeepSWE, beating GPT-5.6 Sol and Fable 5; cost is $0.24 per task.
• Language Performance: GPT-5.6 Sol leads in four languages including Python and Go; Fable 5 dominates Rust (85%); DeepSeek-V4 Pro shows stability without collapsing.
• Behavior Correlation: DeepSeek-V4 Pro and Fable 5 have the lowest task result correlation (0.39), indicating high diversity; Pro is most similar to Sol.
• Routing Strategy: A two-stage routing strategy using Pro first and upgrading to Sol only if tests fail yields an 83.0% solve rate at $3.35 per task.
• Independent Review: GitHub user xiaobright found that the official DeepSeek V4 Pro shows significantly improved scores under specific configurations.
• Stop Control Issue: @karminski3 confirmed via 100M token tests that the "stop control issue" is independent of scaffolding and prompt changes.
• Real-world Case: In a programming challenge shared by DHH, DeepSeek Pro V4 Max cost $23, far less than Fable's $550.

Unconfirmed

• Version Naming: Some posts refer to the model as DeepSeek-V3 (Pro 0813); it is unconfirmed if this is the same model as DeepSeek-V4 Pro.

Why it matters

• The combination of 88.5% pass@4 and a $0.24 cost poses a direct value challenge to GPT-5.6 Sol and Fable 5, potentially influencing engineering team model selection.
• The low correlation (0.39) between Pro and Fable suggests task coverage complementarity, and the two-stage routing strategy demonstrates how to leverage this for optimized cost and performance.

2026-08-15 ~ 2026-08-16 · 10 related posts

• [source] DeepSeek-V3 Beats GPT-4o and Claude 3.5 Sonnet on DeepSWE Benchmark — zainhas · 2026-08-15
• [source] DeepSeek-V4 Pro beats Sol and Fable in coding tasks — zainhas · 2026-08-15
• DeepSeek-V4 Pro coding analysis: Stable but weaker on Rust — zainhas · 2026-08-15
• DeepSeek-V4 Pro and Fable show lowest task correlation — zainhas · 2026-08-15
• Optimized model routing: DeepSeek first, cascade on failure — zainhas · 2026-08-15
• DeepSeek V4 Pro tops benchmarks with specific configs, rivaling GPT-5.6 and Claude — teortaxesTex · 2026-08-15
• DeepSeek V4 Max Completes Challenge with Just $23 Cost — burny_tech · 2026-08-16
• DeepSeek V4-Pro hands-on: Is max reasoning worse than high? Can Harness save it? — karminski3 · 2026-08-16
• [source] DeepSeek V4-Pro's stop-control issue persists across scaffolds; 100M token test shows prompts don't fix it — karminski3 · 2026-08-16

1 near-duplicate retellings: zainhas