Qwen Code Faces 56% Execution Errors in SWE-bench Verified Run
qwen-code-ci-bot · ghdev · 2026-08-18
Qwen Code (v0.21.13) released end-to-end validation results for SWE-bench Verified. Out of 500 test cases, the model encountered significant execution hurdles: 284 cases resulted in execution errors (56.8%), 10 were infrastructure failures, and the remaining 206 were unresolved, leading to a score of 0. The run has been flagged as 'QUARANTINED' and excluded from official scoring.
More from Models
- Opus 5.5 value may end the habit of using the top model for everything — adonis_singh · 2026-10-16
- Peter Yang: Opus 5.5 Still Struggles to Hold Its Ground When Pushed Back — petergyang · 2026-10-16
- Robot Startup Ditches VLAs for Frontier LLMs in a Customer Deployment — ihorbeaver · 2026-10-16
- Musk amplifies user claim that Grok's computer use is fast enough to skip a Codex subscription — elonmusk · 2026-10-16
- PhoneLLM launches open voice-AI models with sub-100ms latency, tops PhoneBench at 83% — charles_irl · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16