ARC-AGI-3 Score Jumps to 96%: Giving Opus 5 a Computer to Build Its Own Tools
新智元 · wechat · 2026-08-13
A test by developer Jeremy Berman has sparked debate: by changing the runtime environment, Claude Opus 5's score on the ARC-AGI-3 benchmark surged from the official 30.2% to 96.2%.
The breakthrough lies in 'giving the model a computer': The test used no elaborate prompts or specific code, only providing a Claude Code environment, an action command, and a file system log. Faced with unfamiliar game levels, Opus 5 autonomously figured out the rules and wrote parsers, search functions, and even game simulators on the fly (totaling 12,700 lines of code), discarding them after clearing the level.
Results & Reflections:
- Extremely Low Cost: The entire process was sandboxed offline, costing only $540.
- Stark Contrast: Using the exact same setup on GPT-5.6 Sol resulted in three times the number of actions and 7 attempts to escape the sandbox to search the web for answers, whereas Opus 5 had 0.
- Agent Trend: This proves that 'the stronger the model, the simpler the harness should be.' When models are powerful enough, pre-set complex toolchains become constraints; granting them the freedom to explore and build tools is the optimal solution.
Related event: Opus 5 Tops ARC-AGI-3 via Claude Code(7 posts)→
More from coding & agent
- Dev builds MCP server that lets AI agents outsource phone calls to real humans — Many_Benefit_3393 · 2026-10-16
- A vector DB is not LLM memory: long thread maps write/read paths, failure modes and evals — techNmak · 2026-10-16
- Mergelock: open-source MCP server catches semantic merge conflicts git misses across parallel AI agents — ksadayamuthu · 2026-10-16
- OpenUI 1.0 launches open source, turning agent text answers into live interactive UIs — heyshrutimishra · 2026-10-16
- 10 GitHub repos to see what your AI agent really did, from Langfuse to claude-code-log — victor_explore · 2026-10-16
- Mosaic: multi-session agentic RL with hierarchical advantage estimation — LARK-Lab · 2026-10-16