27,280 coding-agent trials show the best harness changes with every model
shoumikgoswami · reddit · 2026-11-18
Runta and Baseten tested 22 coding-agent harness configurations across five models in 27,280 trials. Key finding: switching models changes which harness offers the best value — Codex CLI had the lowest cost per passed task with GLM-5.3 ($0.803) but ranked only 12th with Muse. A model upgrade is a reason to re-test your harness, not carry over the old setup.
Stability varies widely: on repeated resampling of their 62 tasks, Codex CLI + GLM-5.3 kept winning in 99.6% of samples, while the other four models' winners held their lead in only 25-53%.
Caveat: this is a vendor-run benchmark (Runta sells agent runtime infrastructure) — treat it as a prompt to test your own pairing, not a shopping list.
More from coding & agent
- Draftmux: an open-source desktop editor for polishing AI-written text with Claude Code — rair41 · 2026-11-18
- Dev wants a routing model to dispatch intents between Codex and Claude MCPs — TheZachMueller · 2026-11-18
- One dev ships full zombie survival game built by a team of Hermes AI agents — Teknium · 2026-11-18
- 10 ChatGPT + Codex prompts to diagnose and fix a slow laptop yourself — anthara_ai · 2026-11-18
- Agent auto-iterates Triton kernels in background, beats all OSS implementations on Strix Halo — qtnx_ · 2026-11-18
- Microsoft's Autopilot Agent Is Built on OpenClaw, Now in Preview for Frontier Customers — pswider · 2026-11-18