DeepSWE benchmark results met with skepticism as community calls for third-party verification

DeepSWE's widely circulated benchmark results drew skepticism from X user scaling01, who suggested such scores often reflect benchmark gaming and urged waiting for third-party verification. Datacurve responded that all DeepSWE problems and scoring scripts are publicly available.

2026-10-18 ~ 2026-10-18 · 2 related posts

• DeepSWE benchmark skeptics told to wait — Datacurve points to fully public repo — scaling01 · 2026-10-18
• scaling01: If Grok and Gemini match frontier models, they're probably benchmaxxed — scaling01 · 2026-10-18