New SWE-bench Multimodal leaderboard: Claude Opus 5.5 leads at 70.21% on visual bugs
OfirPress · x · 2026-10-15
Vals AI launched a leaderboard for SWE-bench Multimodal, a harder SWE-bench variant focused on visual bugs (maps not rendering, buttons on the wrong side). Claude Opus 5.5 leads at 70.21% accuracy ($4.92/test), ahead of Claude Fable 5.1 (61.25%) and Claude Sonnet 5.5 (58.33%); Gemini 4 Argon hits 56.04%, and DeepSeek V4.1 Flash offers standout value at 46.46% for $0.23/test. Even on this relatively simpler bench, much work remains.
Related event: SWE-bench Multimodal Leaderboard Launches, Claude Opus 5.5 Leads(4 posts)→
More from Models
- Google can't rule out Loss-of-Control threshold for Gemini 4 Argon, table shows — TheZvi · 2026-10-15
- Indra OCR Fact-Check: Accuracy Claims Hold, But the 17x Speed Claim Is Really ~3x — TheOyinbooke · 2026-10-15
- Frontier AI duopoly cracking as Google and Meta ship new models — danshipper · 2026-10-15
- GPT-6 Astra solves 49 of 50 unpublished physics problems at 98%, catching physicists' errors — 141_1337 · 2026-10-15
- Rejected by NeurIPS: study shows a single test-set replica beats uncontaminated irreducible error — RylanSchaeffer · 2026-10-15
- Google says Gemini 4 Argon is delivering breakthroughs in quantum computing, fusion and protein design — GoogleDeepMind · 2026-10-15