HarnessDev Benchmark: LLMs Still Lag Humans at Building Their Own Agent Harnesses
_akhaliq · x · 2026-09-04
HarnessDev is a new benchmark testing whether LLMs can create and iteratively improve the agent harnesses that shape downstream performance.
- Spans 6 creator LLMs, 4 domains, and 5 benchmarks
- Generated harnesses show promise but still lag mature human-engineered systems in code, search, and research
- Evolution gains are often unstable and depend on the runtime model
Project page: self-developing-agents.github.io
More from coding & agent
- Developer builds himself a 'Rork MoTS' agent on Grok Bot, aiming for full autonomy before GTA 6 — rudrank · 2026-10-21
- OpenAI Live API tip: control voice sessions from your backend via sideband WebSocket — pbbakkum · 2026-10-21
- OpenAI's MultiLangBench: PHP scores highest (91.3%) at building a database engine from scratch — ricklamers · 2026-10-21
- UserProof lets Claude Code watch your user session recordings and boost conversion for $29/mo — doooyle · 2026-10-21
- dphnAI's distributed inference network served 32B DeepSeek tokens in 24h, ~6% of Venice's daily volume — Ronangmi · 2026-10-21
- Superconductor revamps agent picker spanning 8 coding agents with search and favorites — sergeykarayev · 2026-10-21