28,849-trial Harness Leaderboard: bash-only mini-swe-agent beats openclaw 51.4% vs 27.3%
bansalg_ · x · 2026-11-06
joelniklaus released a Harness Leaderboard built on 28,849 trials ($4,746 of inference), holding model/provider/reasoning effort fixed with 3 attempts per task, comparing 9 coding-agent harnesses on open models (GLM-5.3-Flash, DeepSeek-V4.1-Flash, Flint on Terminal-Bench 2.1 and DeepSWE). Key finding: bash-only mini-swe-agent solves 51.4% of tasks on average vs openclaw's 27.3%, with four other harnesses statistically tied. OfirPress notes mini-swe-agent targets fully autonomous task-solving while Claude Code/Codex are human+AI collaboration assistants, so many CC/Codex features aren't needed for benchmarks.
Related event: 28,849 Tests Rank 9 Coding Agent Harnesses(2 posts)→
More from coding & agent
- Conductor shuts down after Meta acquisition; cherry π ships a one-command migration path — sull · 2026-11-06
- LangChain says most of its internal agents rely on async subagents, now in deep agents — Hacubu · 2026-11-06
- SaaStr runs Replit Agent as its 'CMO': 3 marketing ideas a day — amasad · 2026-11-06
- Visa's Trusted Agent Protocol uses passkeys to verify AI shopping agents at checkout — NVIDIAAI · 2026-11-06
- Cohere's North platform secures AI agents across cloud, private and air-gapped deployments — NVIDIAAI · 2026-11-06
- Choosing browsers by how easily Codex can control them — TheZachMueller · 2026-11-06