I Analyzed 246 Repos and 57 Papers on Agent Harnesses — Here's What Actually Works

Marmelab · reddit · 2026-09-25

The author reviewed 246 open-source repos and 57 papers to separate emerging best practices in agent harness engineering from hype.

TLDR: the best harnesses are small, evidence-based, and constantly refined.

What works:

Write the harness yourself (or have another human do it): An ETH Zurich study found machine-generated context reduces task success versus no context at all while raising inference cost 20%; human-written context improved success 4%.
Build the harness in response to observed failures: modern agents are already capable; the harness should compensate for real weaknesses.
Limit tools and skills exposed to the agent: Vercel removed 80% of tools and success jumped from 80% to 100% with half the tokens; Microsoft cut 100 tools to 2.
Change one component at a time and measure: otherwise you can't tell which change helped.
Have both tests and evals: AGENTS.md, hooks, and scripts can simply be ignored.
Test both "should happen" and "should NOT happen" cases: one-sided evals create one-sided optimization, and the harness drifts toward blocking everything.

More from coding & agent