Claude Fable 5.1 hits 90% on ARC-AGI-2 semi-private at $4.49 per task
geoffwolfe · x · 2026-09-19
- ARC Prize's page shows Claude Fable 5.1 scoring 97.5% on ARC-AGI-1 Semi-Private at $1.40/task and 90.0% on ARC-AGI-2 Semi-Private at $4.49/task at max reasoning effort.
- Across five effort levels, ARC-AGI-2 scores scale from 78.3% (Low) to 90.0% (Max), showing consistent gains with inference budget; ARC-AGI-1 is nearly saturated.
- The verified leaderboard also lists variants from DeepSeek V4, Gemini 3.x, GPT-5.6/6, Grok 4.x and Kimi K3.
- Commenters note ARC-AGI-2 has become a score-and-cost benchmark, with remaining signal concentrated in the hard tail that still separates reasoning effort, consistency, and inference budget.
More from Models
- Hindsight ships 0.8B model for agent memory decisions: 2.5x cheaper, 1.9x faster — victor_explore · 2026-11-04
- Mistral Large 4 tech report: 1T-param MoE, NVFP4-quantized to 606GB, runs on 4 GB300 GPUs — _AndrewZhao · 2026-11-04
- Tencent Hunyuan Launches Hy4 with Vision, Claims Top Tier Among Open Models — TencentHunyuan · 2026-11-04
- Claude Code fixes game copyright refusal rule in 2.1.317: naming a game now works — apples_jimmy · 2026-11-04
- Microsoft Phi-4-mini: a 3.8B model with 128K context and 52.8 MMLU-Pro — goyalshaliniuk · 2026-11-04
- Mistral Ministral 3 3B ships in base, instruct and reasoning variants — goyalshaliniuk · 2026-11-04