Harness-of-Harness paper: 52.25% avg gain across 3 benchmarks, builds playable FPS unattended
aigclink · x · 2026-09-05
The Harness-of-Harness paper (Shanghai AI Lab) details a meta-framework organizing coding-agent harnesses into planning-coding-testing loops with small verifiable increments and separated evaluation.
- Evaluated on GameCraft-Bench, FrontierSWE, and ProgramBench with Codex+GPT-5.5(high), OpenCode+DeepSeek-V4-Pro, and Pi+MiniMax-M3
- Average relative gain of 52.25% (max 82.86%) after three iterations
- Multi-day, 70+ iteration autonomous build of Fusepoint, a human-playable FPS with coherent story, mechanics, visuals, and audio, from a single PRD
More from coding & agent
- Grady Booch revives Weinberg's 1971 quip: software so buggy one woodpecker could destroy civilization — Grady_Booch · 2026-10-23
- One Agent Replaces Half an SDR Team: Full Lead-Research-to-Email Workflow — tibo_maker · 2026-10-23
- Why don't coding agent harnesses ship with their own sandbox? Only amp does — Aryvyo · 2026-10-23
- Dev builds a VST synth with Codex and MCP, lets the AI jam in his studio — Yamapama · 2026-10-23
- AgentsInTheCloud update: use your Claude subscription with built-in coding agents — lucasmeijer · 2026-10-23
- Phone agents vs. secure screens: banking apps treat helpful agents like malware — ghostmalone02 · 2026-10-23