Harness-of-Harness carries state across runs: 71.52 vs 58.24 on long-horizon coding
rohanpaul_ai · x · 2026-09-06
Harness-of-Harness (HoH) tackles the core failure modes of long-horizon coding agents: forgetting earlier decisions, repeating work, breaking working features, and missing unfinished requirements.
- Method: carry the current software, test evidence, known issues, and an updated plan into the next coding run, with independent testing and replanning from real failures.
- Results: improved all 3 tested agent setups across 3 benchmarks. Clearest comparison: Codex + GPT-5.5 on GameCraft-Bench over 3 passes scored 71.52 with HoH vs 58.24 by simply continuing the same agent.
- Stress test: 70 loops produced a playable FPS from high-level requirements — though only one game project with extra tools, so generalization remains open.
- Takeaway: for long-running agents, invest in persistent project state, independent QA, and evidence-driven replanning — not just more calls.
More from coding & agent
- 105 Real Bugs Tested: GPT-6.1 Sol Over 10x Cheaper Than GPT-5.6 Sol — PawelHuryn · 2026-10-23
- Dev builds SkiFree-style sled game in one prompt, praises DeepSeek Flash — victormustar · 2026-10-23
- Every ditched personal agents for a shared Claude Managed Agents coworker in one week — every · 2026-10-23
- WorkOS shows how to let AI agents safely sign up for your app with auth.md and ID-JAG — AI Engineer · 2026-10-23
- Devs on Reddit swap tactics for stopping prompt regressions in AI agents — SleepFlimsy6963 · 2026-10-23
- Claude Code + ComfyUI MCP workflow for client work: paid models or open weights? — newaccount47 · 2026-10-23