Claude Opus 5.5 scores 18.5% on ProgramBench, up from 0% at May launch
winer666 · reddit · 2026-10-28
Per Jack Clark's Import AI newsletter (Oct 26), Claude Opus 5.5 (xhigh) now resolves 18.5% of ProgramBench's 200 from-scratch rebuild tasks — rewriting programs like SQLite, FFmpeg and ripgrep from a compiled binary plus docs. Every model scored 0% at the benchmark's May launch. Clark predicts (his own forecast, not a measured result) saturation by end of 2027. The poster raises two open questions: does rebuild-from-binary mainly test reverse-engineering and long-horizon persistence rather than real software engineering? And what benchmark should replace it once it saturates?
More from coding & agent
- Codex daily build series day 23: skipping iCloud plugins for a reset — dotey · 2026-10-28
- Cloudflare Access adds fully customizable pages with HTML + Liquid templates — dinasaur_404 · 2026-10-28
- Opus retried every session 20.6 million times during a memory leak hunt — johnlindquist · 2026-10-28
- Open-Source Skill Teaches AI Agents to Design and Produce Videos — songguoxiansen · 2026-10-28
- Not Everything Needs an Agent: Fixed Steps Are Recipes, Save Agents for Judgment — adamse · 2026-10-28
- Dev suggests self-hosting via Tailscale only: no open ports, tailnet as sole access path — vincesitzmann · 2026-10-28