Claude Opus 5 Tops ProgramBench with Original Coding at High Cost

Claude Opus 5 (xhigh) secured first place on the ProgramBench benchmark, demonstrating formidable coding capabilities by rebuilding complete programs from scratch, though its high usage costs have also sparked concerns.

已确认

• In the ProgramBench evaluation, which requires rebuilding complete programs like sqlite and ffmpeg from scratch, Claude Opus 5 (xhigh) took first place by solving 9 tasks (4.5%).
• The model achieved record highs across various metrics, with the number of complete passes (100% test pass rate) doubling from 2 in the past to 4.
• Developer testing revealed that Opus 5 perfectly reconstructed 9 open-source projects. Comparisons via diff against the actual upstream repositories confirmed that the code was not simply copied but featured novel solutions.
• When testing the --to-ascii feature of the rnr tool in an offline environment, Opus 5 exhibited autonomous reasoning and exploration skills, proactively sending queries to the tool to deduce the character mapping table on its own.
• The model incurs extremely high costs during coding tasks. A single complex task cost $50.53, involving 257 rounds of conversation interactions and outputting over 357,000 tokens, far exceeding previous models in the same series.
• The ProgramBench benchmark was jointly launched by Meta's Superintelligence Lab, Stanford, and Harvard. The complete leaderboard, per-task costs, and execution trajectory data have been released.

为什么重要

• These test results prove that the boundaries of AI capabilities in complex, original software engineering tasks are expanding significantly. It can now handle not just surface-level code issues but also autonomously construct underlying logic.
• Despite its astonishing capabilities, the cost of over $50 per task means that commercializing and popularizing this extreme coding ability still faces a steep economic threshold at this stage.

2026-08-13 ~ 2026-08-13 · 6 related posts

• [source] Claude Opus 5 Tops ProgramBench, Significantly Beating GPT-5.6 — jyangballin · 2026-08-13
• Claude Opus 5 Tops ProgramBench, Doubling Fully Resolved Tests — jyangballin · 2026-08-13
• [source] Testing Claude Opus 5: Perfectly Rebuilds 9 Open-Source Projects with Original Code — jyangballin · 2026-08-13
• Claude Opus 5 Rebuilds Million-Character Mapping Table Offline in 6 Seconds — jyangballin · 2026-08-13
• Claude Opus 5 Hits Record High Costs: Over $50 for a Single Task — jyangballin · 2026-08-13
• [source] ProgramBench Eval: Claude 3 Opus Incurs a Staggering $50 Cost Per Task — jyangballin · 2026-08-13