GPT-6 Sol scores 89.6% on ARC-AGI-2 but only 23% on ARC-AGI-3, ARC Prize reports
fchollet · x · 2026-09-29
ARC Prize published verified ARC-AGI results for OpenAI's GPT-6 Sol:
- ARC-AGI-3: 4.6% ($5.6K) on the standard harness, 23.0% ($8.7K) with the provider adapter harness — far behind Astra's 99.9%, though well above Luna's 0.59%
- ARC-AGI-2: 89.6% at $0.44/task
- ARC-AGI-1: 95.5% at $0.14/task
Sol effectively solves ARC-AGI-2, but the huge gap on ARC-AGI-3 — which tests open-world exploration and reasoning — shows frontier models still struggle to generalize in genuinely novel environments.
More from Models
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Top LLM score more than doubled since GTA VI trailer dropped — ApprehensiveAd3629 · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- AI daily: DeepSeek rate-limits chats, Qwen haggling spurs shop warnings, Doubao workspace launch — 快鲤鱼 · 2026-11-16