246 repos and 57 papers later: what actually works in agent harness engineering
Marmelab · reddit · 2026-09-25
Analyzing 246 open-source repos and 57 publications on coding-agent harnesses, the author distills emerging best practices: harnesses should be small, evidence-based, and constantly refined. Key findings: an ETH Zurich study shows machine-generated context reduces task success vs. no context (while raising inference cost 20%), while human-written context adds 4%; build harnesses only in response to observed failures; limit exposed tools (Vercel cut 80% of tools and success jumped 80%→100% with half the tokens; Microsoft went from 100 tools to 2); change one component at a time and measure; maintain both tests and evals since instructions like AGENTS.md can simply be ignored; and test both "should happen" and "should NOT happen" cases—one-sided evals silently drift toward blocking everything and strangling the agent.
Related event: Analysis of 246 Repos and 57 Papers: Effective Agent Harnesses Stay Small(2 posts)→
More from coding & agent
- Perplexity CEO shows agent building browser-playable 3D zombie shooter in hours with zero 3D assets — AravSrinivas · 2026-11-11
- The complete harness engineering roadmap: how OpenAI shipped ~1M lines with zero hand-written code — blaizedsouza · 2026-11-11
- Micius-Flash: 27B Multimodal Model for Hardware and Semiconductor Agents — zhiman-ai · 2026-11-11
- Viral 'the harness is the machine' thread touts unverified Claude Fable/Mythos 5.2 9-equation architecture — blaizedsouza · 2026-11-11
- Compaction creates a new failure mode for coding agents: critical bug details can vanish mid-fix — blaizedsouza · 2026-11-11
- What 40+ coding-agent harnesses do between your prompt and the model, dissected — blaizedsouza · 2026-11-11