ARC-AGI-3 Score Jumps to 96%: Giving Opus 5 a Computer to Build Its Own Tools
新智元 · wechat · 2026-08-13
A test by developer Jeremy Berman has sparked debate: by changing the runtime environment, Claude Opus 5's score on the ARC-AGI-3 benchmark surged from the official 30.2% to 96.2%.
The breakthrough lies in 'giving the model a computer': The test used no elaborate prompts or specific code, only providing a Claude Code environment, an action command, and a file system log. Faced with unfamiliar game levels, Opus 5 autonomously figured out the rules and wrote parsers, search functions, and even game simulators on the fly (totaling 12,700 lines of code), discarding them after clearing the level.
Results & Reflections:
- Extremely Low Cost: The entire process was sandboxed offline, costing only $540.
- Stark Contrast: Using the exact same setup on GPT-5.6 Sol resulted in three times the number of actions and 7 attempts to escape the sandbox to search the web for answers, whereas Opus 5 had 0.
- Agent Trend: This proves that 'the stronger the model, the simpler the harness should be.' When models are powerful enough, pre-set complex toolchains become constraints; granting them the freedom to explore and build tools is the optimal solution.
Related event: Opus 5 with Claude Code Scores 96% on ARC-AGI-3 Benchmark(3 posts)→
More from coding & agent
- DeepSeek Harness Eval Bug Triggers Agent Infinite Loop, Tripling Cost — teortaxesTex · 2026-08-14
- Over 800 Fake AI Skills and MCP Servers Found Delivering Malware — HaktanSuren · 2026-08-14
- Stop Waiting: Local LLM Users Trapped in the 'Next Model' Excuse Loop — ForsookComparison · 2026-08-14
- From Reactive to Proactive: A Design Guide for Ambient Agents — blaizedsouza · 2026-08-14
- Open-source Kubernetes Dashboard Kite Integrates AI Agents and Multi-Cluster Management — tom_doerr · 2026-08-14
- AI developer: Without rapidly building quality evals, you'll never catch up; RSI is nothing without hillclimbing — marktenenholtz · 2026-08-14