CMU paper: a 4B model rewrites an agent's harness and beats the 35B model it learned from
irinarish · x · 2026-10-14
CMU researchers trained a small 4B model to rewrite an AI agent's harness — the program around the model covering how it gets called, which tools run, and where information flows — and it beat the 35B model it learned from. The 4B model reads a report from the last run, proposes harness edits, and keeps the best-scoring one; the task model itself never changes.
Key findings:
- On unseen task types, edit quality rose from 0.32 to 0.62
- 5 sequential edits matched or beat the best of 8 parallel tries
- Broken edits dropped from 38% to 15%
The thread also outlines a reproducible loop: put your harness in one file (prompts, steps, tool calls), split tests into learn/score/final sets, run on the learn set, save a failure report, and ask a model to propose edits. The linked article details six decisions for designing an agent harness you can leave alone.
More from coding & agent
- LlamaIndex benchmarks Gemini 4 Argon vs Muse models on hard document OCR — llama_index · 2026-10-14
- AI reality-check audit of 50+ projects: FrankenRedis report digs to root causes — doodlestein · 2026-10-14
- Building savi: why coding agents should jam like jazz, not disappear for an hour — TAbrodi · 2026-10-14
- AI agents hit monorepo scale: you need Google-style copy-on-write primitives, not Git — rickasaurus · 2026-10-14
- MCP tool lets AI agents call you on WhatsApp when they get stuck on a decision — Augusto_pinheiro · 2026-10-14
- Argon tops hallucination bench with >50% false claim rate, prompting Gemini users to back off — brandon_galang · 2026-10-14