sudo L7 benchmark: Claude Opus 5.5 matches GPT-6.1 on code, leads 23 points on judgment
echen · x · 2026-10-21
A key finding from Surge AI's sudo L7 benchmark: Claude Opus 5.5 and GPT-6.1 Sol are essentially tied on functional correctness (82.1% vs 82.4%), but Opus dominates on staff-level behaviors — 76.1% vs 53.0% on "thought partnership" (suggesting better approaches than requested) and 66.4% vs 49.4% on communication (explaining decisions, tradeoffs, limitations). Sonnet 5.5 also beats both GPT-6 variants on communication (72.4%). The author's point: evaluating coding agents by passing tests is like promoting engineers because their unit tests pass — sudo L7 measures the judgment that actually gets you to staff level, using 60 tasks grounded in real production repositories.
More from coding & agent
- How a solo founder hit £100K/month in 95 days with a Claude Code system and zero hires — aakashgupta · 2026-10-21
- Dev hasn't touched his MacBook screen in weeks: agents do everything, iPad is the interface — MarcJSchmidt · 2026-10-21
- Reason Network OSS lets Claude Code, Codex and hosted agents like Grok Bot talk peer-to-peer — AIFlow_ML · 2026-10-21
- Omnigent v0.19 adds MCP gateway, keyless sandbox Git access and SDK reasoning controls — matei_zaharia · 2026-10-21
- Sentry Android SDK is 20-30% faster, saving 6,250 CPU-hours per day — zeeg · 2026-10-21
- mcporter bridges MCP into non-MCP agent harnesses, dev shows on-the-go workflow — steipete · 2026-10-21