ProgramBench leaderboard: agents rebuild programs from binaries, Opus 5.5 tops at 18.5%
TheReedemer69 · reddit · 2026-11-15
The attached image shows the ProgramBench leaderboard (200 tasks, updated Oct 22) from Meta Superintelligence Labs, Stanford and Harvard: given only a compiled binary and docs, agents must architect and implement a full codebase reproducing the original program. Claude Opus 5.5 (xhigh, mini-SWE-agent) leads with 18.5% resolved (70.5% almost, $36.36 cost); Claude Fable 5.1 hits 9.0% ($56.84), Meta's Muse Wonder Full 8.5%, Muse (xhigh) 5.5%, GPT 6 Astra (max) 5.0% ($11.47). The poster asks for predictions on Fable 5.2.
More from coding & agent
- Meituan open-sources LongCat-2.5, a 1.6T agent model — deploy guide is NPU-only — javimbuilds · 2026-11-15
- Cursor agent deleted uncommitted files in 24/24 runs despite skill instructions — InterestingCarrot997 · 2026-11-15
- terrio v0.9.0: pure-Rust geospatial I/O with no GDAL install, MCP for AI agents — giswqs · 2026-11-15
- 5 terminal-first tips from a Claude researcher for becoming an AI power user — devanshmehta · 2026-11-15
- A four-tool pipeline to create playable GTA 6-style AI 3D characters — filiksyos · 2026-11-15
- Review velocity is the new bottleneck as AI coding pushes review time toward zero — Elijah_Meeks · 2026-11-15