Opus 5 appears to improve on ARC-AGI 1 and 2, and may rely on algebraic puzzle solving
herbiebradley · x · 2026-07-25
Herbie Bradley says the Opus 5 system card shows clear gains on ARC-AGI 1 and 2 compared with 4.7/4.8.
He also notes that the system-card description of an ARC-AGI-3-like task sounds like the model learns to turn a visual puzzle into explicit algebra, which he sees as a very trainable strategy. Based on that, he predicts the improvement likely came from a focused research project on these puzzle types, possibly using a data vendor under an exclusivity agreement.
Related event: Claude Opus 5 Sets New ARC-AGI-3 Record(15 posts)→
More from Models
- Cohere sandbox: Qwen model tried to upload probe file to Beijing cloud server unprompted — simonguozirui · 2026-10-27
- Kimi K3.1 Pro tops open-weight models on FrontierML benchmark, still far behind Opus 5.5 — teortaxesTex · 2026-10-27
- Across 7 tasks, highest reasoning setting isn't the best for most open models — keunwoochoi · 2026-10-27
- Open Models Match GPT-6 on MedMCQA but Trail 17 Points on Hard Clinical Cases — keunwoochoi · 2026-10-27
- Health-Focused Benchmark Reranks LLMs: Muse Spark Open Jumps from #9 to #1 — keunwoochoi · 2026-10-27
- Rumors: DeepSeek delayed on agents, losing its AI4Math edge as Kimi K3 breaks through — teortaxesTex · 2026-10-27