GitHub publishes harness-level data comparing Copilot CLI vs Codex and Claude Code
0xkarasy · x · 2026-11-18
GitHub now publishes data on how the GitHub Copilot harness performs relative to other harnesses including Codex CLI and Claude Code CLI, supplementing existing model benchmarks on Terminal Bench 4.0 and DeepSWE 1.1. The chart plots resolution rate vs. avg cost per task: Claude Opus 5.5 and GPT-6 Astra lead (45-50% at $2.5+), GPT-6.1 Sol hits 47% at $0.5, Claude Sonnet 5.5 28%, while GPT-6 Luna is cheapest but under 10% resolution. Harness choice matters less than model choice.
More from coding & agent
- Pipecat open-sources Audience: a tiny classifier so voice agents know when they're being addressed — alexcovo_eth · 2026-11-18
- Giving Your Agent Your Own API Token: An Infra Engineer's Cautionary Tale — WesternNo2429 · 2026-11-18
- Weave: Model Writes One Script, Rust Runtime Parallelizes Independent Tool Calls — Business_Rain_1042 · 2026-11-18
- Microsoft open-sources Work IQ Starter Pack to fill demo tenants with realistic data — DanWahlin · 2026-11-18
- Cloning $50k agency-built WebGL landing pages with Claude, just for fun — mickeyxfriedman · 2026-11-18
- Basis builds a formally verified undo button for Amazon S3 to roll back agent mistakes — xuanalogue · 2026-11-18