Opus 5 reaches about 30% on ARC-AGI-3 in a cost-versus-skill chart
Dr_Singularity · x · 2026-07-25
The post highlights Opus 5 on ARC-AGI-3 with an attached chart titled “Novel problem-solving by cost.”
Key points from the image:
- Opus 5 (high) scores about 30%
- Opus 4.8 (high) is around the low single digits
- GPT-5.6 Sol appears in the high single digits at a higher evaluation cost
The chart emphasizes the cost-versus-performance tradeoff on a benchmark aimed at novel problem solving, not just standard benchmark completion.
Related event: Anthropic Releases Claude Opus 5: SOTA Performance at Half the Price(128 posts)→
More from Models
- MissionReport benchmark: only 3 of 540 AI-generated government reports pass all criteria — Exp_Mark · 2026-10-27
- Local beats cloud: Qwen 3.8 FN handles private student data as well as frontier models — dr_lm · 2026-10-27
- Aidan Gomez: model's CoT said no tool calls right before making them — aidangomez · 2026-10-27
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27