Surge AI benchmarked coding-agent harnesses: vanilla Claude Code beat them all
echen · x · 2026-11-05
After Garry Tan revealed he spends $200K/year on tokens running Claude Code with 23 skills covering CEOs, designers, and eng managers, Surge AI's data science team asked: do elaborate coding-agent setups actually help?
They benchmarked popular setups using Opus 5.5 in Claude Code on Surge's sudo L7 software engineering benchmark: vanilla Claude Code, Superpowers (a plan-and-test skills plugin with 311K+ GitHub stars), and Compound Engineering's plan/work/review workflow, among others.
Verdict: raw model intelligence beat all the extra machinery. In one case, a harness with 37 additional skills spawned 17 review agents for a trivial Zoom-link change — five reviewing the plan, five hunting simplifications, seven reviewing code — yet none noticed the central product flaw: confirmation emails went out before payment, leaking the Zoom passcode to unpaid registrants. The team plans to test gstack next.
Related event: Garry Tan's $200K Claude Code Setup With 23 Skills Falls Short in Tests(3 posts)→
More from coding & agent
- Open-source iRacing telemetry lib measures understeer per corner phase with zero AI — JeremyCMorgan · 2026-11-05
- Engram blog: RL training complements text memory, stays robust through millions of token edits — canaesseth · 2026-11-05
- Agent Tool 'Dot' Lets Users Send Feature Requests Directly to the Dev Team — dkundel · 2026-11-05
- Model gets stuck in a loop spamming "it failed!!!" — burning 140K tokens — CtrlAltDwayne · 2026-11-05
- AX to support migrating agentic workloads between local and production clusters — rakyll · 2026-11-05
- OpenAI upgrades Codex: marked-up images now pinpoint edits via API — OpenAIDevs · 2026-11-05