Grok 4.5 scores 85.7% on ARC-AGI-1 but just 0.3% on ARC-AGI-3
scaling01 · x · 2026-07-24
Grok 4.5 posts strong ARC-AGI scores, but still collapses on ARC-AGI-3
ARC Prize’s verified results for Grok 4.5 show a sharp split across benchmarks:
- ARC-AGI-1: 85.7% at $0.33/task
- ARC-AGI-2: 52.6% at $0.78/task
- ARC-AGI-3: 0.3% at $6.9K/task
The tester also said raising reasoning effort from medium to high did not improve performance or cost.
More from Models
- DeepSeek cut KV cache per token from 389KB to 890B in two years as open-closed gap debate rages — teortaxesTex · 2026-10-27
- Unverified: fable 5.5 speculated at 4-5T parameters to outscale 1-3T Opus — haider1 · 2026-10-27
- Kimi K3 Is 'Open' Until You Succeed: License Caps Free Use at $20M Revenue — alex_verem · 2026-10-27
- Screenshot shows 'GPT-6 Astra Ultra' in ChatGPT model picker — unverified leak or joke — justalexoki · 2026-10-27
- DeepSeek Safety Eval Finds "Over-Attribution": Model Mistakes Ordinary Tasks for Tests — Much_Boysenberry9124 · 2026-10-27
- Cohere flags popular open-source model attempting to upload probe file to Beijing bucket — stanislavfort · 2026-10-27