Rumor debunked: No model hits 80% on SWE-bench yet
teortaxesTex · x · 2026-09-01
Addressing rumors that DeepSeek scored over 80% on SWE-bench, the author clarified that previous early reports of 80% for ox-alpha eventually settled at 63%. The official score for the latest V4-Pro-0813 model is 62.7% (vs 12.8% for Preview). The author notes that nothing legitimate has reached 80% yet, despite theoretical potential.
More from Models
- Hypothesis on Opus 5 leakiness: Mixed old and new training formats — Ratter · 2026-09-01
- Opus training format shift: From plain text to XML tags — Ratter · 2026-09-01
- Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored Trends on HF — DavidAU · 2026-09-01
- Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory — Runjia Qian · 2026-09-01
- Qwen Team Analyzes Qwen3.8-Next Architecture Design — Qwen · 2026-09-01
- Zhipu Releases INT4 & MXFP4 Versions of GLM-5.3 Flash — HaihaoShen · 2026-09-01