Surge AI's sudo L7: 60 real-repo tasks testing if coding agents act like staff engineers
echen · x · 2026-10-21
Surge AI details sudo L7, its benchmark for staff-level software engineering: 60 tasks authored by professional engineers, grounded mostly in private production repositories from real companies plus open-source projects. The thesis: today's coding agents can be technically right but professionally wrong — a password manager showing saved passwords in plaintext, a migration deleting records without rollback, a dashboard simulating prices without disclosing it. Junior engineers follow tickets; staff engineers notice what tickets forget, anticipate consequences, and know when passing tests isn't the end of the story. The post links the earlier finding that Opus 5.5 and GPT-6.1 Sol tie on correctness (82.1% vs 82.4%) but Claude leads by 20+ points on thought partnership and communication.
More from coding & agent
- mcporter bridges MCP into non-MCP agent harnesses, dev shows on-the-go workflow — steipete · 2026-10-21
- Chollet: Engineers Hit Record Numbers as Agents Write ~100% of Code, Bottlenecks Remain — manuelkoelman · 2026-10-21
- Dev finds Claude subscription prompts get billed as extra usage unless prompt starts with official header — lucasmeijer · 2026-10-21
- LangChain explains how Deep Agents fight context rot with five mechanisms — LangChain · 2026-10-21
- LangChain answers the most common questions about agent evals — LangChain · 2026-10-21
- uivoid turns any existing API into an authenticated hosted MCP server for free — Necessary_Client_114 · 2026-10-21