FrontierCode 1.1 leaderboard: Claude Opus 5.5 tops at 52% as leading models fail 40%+ of tasks
ArtificialAnlys · x · 2026-10-13
Artificial Analysis launched a leaderboard for Cognition's FrontierCode 1.1, a private agentic coding benchmark testing whether models produce changes real open-source maintainers would merge:
Design
- Dataset built with OSS maintainers (40+ hours per task); 100 private tasks scored via a Stirrup reference harness, 3 attempts each, blocking checks fail to zero;
- Beyond unit tests, it penalizes excessive changes, huge diffs, out-of-scope edits, and unhelpful tests.
Key results
- Claude Opus 5.5 (max, fallback) leads at 52.2%, GPT-6.1 Sol 49.1%, Claude Fable 5.1 49.0%; nine models from six labs pack 43–52%;
- 9 of 100 tasks were passed by no model in any attempt;
- Best open-weights: MiMo-V2.6-Pro (45.8%), then DeepSeek V4.1 Flash and Kimi K3 at 44%;
- Cost standout: GPT-6.1 Sol at $0.81/task for 49% — near-Anthropic performance at less than a tenth of Opus 5.5's $10.15/task.
Related event: FrontierCode 1.1 Leaderboard Debuts: Opus 5.5 Leads at 52%(4 posts)→
More from coding & agent
- Ramp CTO to Discuss Internal Coding Agent Behind ~75% of Merged PRs — graceisford · 2026-10-13
- Meta Muse gains access to approved company tools via Merge's governed MCP endpoint — shensi · 2026-10-13
- Build your own Jev-style decision model in 15 minutes with open models — Arindam_1729 · 2026-10-13
- NVIDIA open-sources SMLIB, its NURBS-based CAD kernel, after 20-year partnership — Promptmethus · 2026-10-13
- LLMs are System 2: Austin LangChain's framing for pairing fast judgment with slow reasoning — colinmcnamara · 2026-10-13
- A feed of next actions is the only sane way to manage massively parallel agents — _AustinCalvert_ · 2026-10-13