Grok 4.5 scores 85.7% on ARC-AGI-1 but just 0.3% on ARC-AGI-3
scaling01 · x · 2026-07-24
Grok 4.5 posts strong ARC-AGI scores, but still collapses on ARC-AGI-3
ARC Prize’s verified results for Grok 4.5 show a sharp split across benchmarks:
- ARC-AGI-1: 85.7% at $0.33/task
- ARC-AGI-2: 52.6% at $0.78/task
- ARC-AGI-3: 0.3% at $6.9K/task
The tester also said raising reasoning effort from medium to high did not improve performance or cost.
More from Models
- Developer Building a Unified Leaderboard of All Model Benchmark Scores — airesearch12 · 2026-09-11
- Rumor claims Kimi faked performance by serving Claude; DeepSeek new model surprises in evals — realsohamparekh · 2026-09-11
- GPT-5.6 writes well but is instantly forgettable, user complains — BasedRaddka · 2026-09-11
- Opus Refuses Protein Research Codebase Over 'Safety' Concerns, Dev Considers Rolling His Own — josephdviviano · 2026-09-11
- User Hails Unconfirmed 'DeepSeek 4.1 Flash' as an Inflection Point in LLMs — himanshustwts · 2026-09-11
- Terminal Bench v4: GLM-5.3 Leads at 41.9%, Kimi-K3 Underwhelms at 12.6% — Ok_Warning2146 · 2026-09-11