Opus 5 Tops ARC-AGI-3 via Claude Code

Developer Jeremy Berman achieved a 96.2% accuracy (pass@2 99.3%) on the ARC-AGI-3 public benchmark using Claude Code paired with Opus 5 (High Reasoning mode), significantly surpassing the model's official evaluation score of 30.2%. The key to this approach lies in changing the runtime environment: rather than relying on fine-tuned prompts or specialized code, it allows the model to synthesize symbolic world models by writing executable code instantly. ARC founder François Chollet retweeted and endorsed this work, noting that all top solutions on ARC-AGI-3 have adopted this style of "LLM-guided just-in-time symbolic world model synthesis."

Confirmed

• Berman announced that Claude Code + Opus 5 (High) failed only 1 out of 25 ARC-AGI-3 public test questions, achieving 96.2% accuracy and a pass@2 of 99.3% on the second attempt.
• The solution relies almost entirely on providing an executable environment rather than ARC-specific designs, allowing the model to encode understanding of causal mechanisms through code.
• Reports indicate the same model scored 30.2% in official evaluations, jumping to 96.2% after the environmental change.
• François Chollet described this as a powerful method for "LLM-guided just-in-time symbolic world model synthesis" and confirmed that all top ARC-AGI-3 solutions utilize this approach.

Unconfirmed

• Francois Chabard cited a score of 95.5%, slightly edging out the 95.4% human expert baseline. This figure differs slightly from Berman's reported 96.2%, and the specific metric used requires verification.

Why it matters

• The massive score gap between the official evaluation and the self-tooling environment demonstrates the significant impact of agent runtime environments and scaffolding on reasoning benchmarks.
• Chollet's endorsement signals that "coding to synthesize symbolic world models" has become a common path for top-tier ARC-AGI-3 solutions, offering a new direction for LLM general reasoning.
• Some observers argue that agents combining recursive language models with continuous testing frameworks are not mere "wrappers" but can substantially enhance model capabilities.

2026-08-13 ~ 2026-08-15 · 7 related posts

• Claude Code Hits 96.2% on ARC-AGI-3 with Almost Zero ARC-Specific Code — khademinori · 2026-08-13
• [source] ARC-AGI-3 Score Jumps to 96%: Giving Opus 5 a Computer to Build Its Own Tools — 新智元 · 2026-08-13
• Opus 5 scores 96.2% on ARC-AGI-3 with generic agent, near-human performance — inductionheads · 2026-08-14
• Agent Harness Breakthrough: Claude Opus Hits 95.5% on ARC-AGI Beating Humans — ycombinator · 2026-08-14
• [source] ARC-AGI-3 new approach: LLM-guided symbolic world model synthesis, Opus writes 269 programs in one pass — fchollet · 2026-08-14
• Neuro-symbolic world models gain support: top ARC-AGI-3 harnesses use this approach — GaryMarcus · 2026-08-15
• Top ARC-AGI-3 solutions use LLM-guided symbolic world model synthesis — paraschopra · 2026-08-15