I Analyzed 246 Repos and 57 Papers on Agent Harnesses — Here's What Actually Works
Marmelab · reddit · 2026-09-25
The author reviewed 246 open-source repos and 57 papers to separate emerging best practices in agent harness engineering from hype.
TLDR: the best harnesses are small, evidence-based, and constantly refined.
What works:
- Write the harness yourself (or have another human do it): An ETH Zurich study found machine-generated context reduces task success versus no context at all while raising inference cost 20%; human-written context improved success 4%.
- Build the harness in response to observed failures: modern agents are already capable; the harness should compensate for real weaknesses.
- Limit tools and skills exposed to the agent: Vercel removed 80% of tools and success jumped from 80% to 100% with half the tokens; Microsoft cut 100 tools to 2.
- Change one component at a time and measure: otherwise you can't tell which change helped.
- Have both tests and evals: AGENTS.md, hooks, and scripts can simply be ignored.
- Test both "should happen" and "should NOT happen" cases: one-sided evals create one-sided optimization, and the harness drifts toward blocking everything.
Related event: Analysis of 246 Repos and 57 Papers: Effective Agent Harnesses Stay Small(2 posts)→
More from coding & agent
- The decade of agents: simulate real customers to test your agent before launch — EXM7777 · 2026-11-11
- FluentCRM MCP Server lets you manage marketing automation from Cursor or Claude — modelcontextprotocol · 2026-11-11
- Databricks' revised Genie agent tells CEO Ali Ghodsi the unvarnished truth — TechCrunch AI · 2026-11-11
- Elvis Saravia: agents will subsume traditional software — and can even cut code bloat — omarsar0 · 2026-11-11
- Using a third-party inference provider with your coding harness hands it shell access to your machine — TheZachMueller · 2026-11-11
- Splash One: flight combat sandbox built with Opus 5.5 in PlayCanvas — willeastcott · 2026-11-11