Claude Opus 5 Tops ProgramBench Coding Benchmark

Claude Opus 5 has achieved top rankings in the ProgramBench benchmark, solving 9 tasks (4.5%) and doubling the number of fully passed (100% pass rate) tasks to 4. Real-world tests confirm that Opus 5 can perfectly reconstruct 9 open-source projects with highly original code and demonstrate autonomous reasoning by deriving character mapping tables offline. Despite these capabilities, a single complex task costs $50.53, involving 257 turns and over 357,000 output tokens.

Confirmed

In the ProgramBench evaluation requiring reconstruction of complete programs like sqlite and ffmpeg from scratch, Claude Opus 5 (xhigh) took first place by solving 9 tasks (4.5%), doubling the fully passed count to 4.
Developer testing confirmed that Opus 5 perfectly reconstructed 9 open-source projects; diff comparisons revealed the code was not a copy but offered novel solutions.
In an offline environment testing the rnr tool, Opus 5 demonstrated autonomous reasoning by actively querying the tool to derive character mappings.
The cost for a single complex task reached $50.53, involving 257 conversation turns and over 357,000 output tokens, far exceeding previous models.
ProgramBench was released by Meta Super Intelligence Lab, Stanford, and Harvard, providing full leaderboards, cost data, and run traces.

Why it matters

These results demonstrate a significant expansion of AI capabilities in complex, original software engineering, moving beyond surface-level fixes to autonomous logic construction.
The cost of over $50 per task highlights that while impressive, such extreme coding capabilities face high economic barriers for current commercial adoption.

2026-08-13 ~ 2026-08-14 · 7 related posts

Primary sources

Claude Opus 5 Tops ProgramBench, Significantly Beating GPT-5.6 — jyangballin · 2026-08-13 22:14 (UTC+08:00)
Testing Claude Opus 5: Perfectly Rebuilds 9 Open-Source Projects with Original Code — jyangballin · 2026-08-13 22:14 (UTC+08:00)
Claude Opus 5 Rebuilds Million-Character Mapping Table Offline in 6 Seconds — jyangballin · 2026-08-13 22:14 (UTC+08:00)

[source] Claude Opus 5 Tops ProgramBench, Significantly Beating GPT-5.6 — jyangballin · 2026-08-13
Claude Opus 5 Tops ProgramBench, Doubling Fully Resolved Tests — jyangballin · 2026-08-13
[source] Testing Claude Opus 5: Perfectly Rebuilds 9 Open-Source Projects with Original Code — jyangballin · 2026-08-13
[source] Claude Opus 5 Rebuilds Million-Character Mapping Table Offline in 6 Seconds — jyangballin · 2026-08-13
Claude Opus 5 Hits Record High Costs: Over $50 for a Single Task — jyangballin · 2026-08-13
ProgramBench Eval: Claude 3 Opus Incurs a Staggering $50 Cost Per Task — jyangballin · 2026-08-13

1 near-duplicate retellings: jyangballin