JHU/CMU paper: a 4B model learns to rewrite agent harness code, beating its 35B teacher
rohanpaul_ai · x · 2026-10-06
A new Johns Hopkins / Carnegie Mellon paper shows a small model can learn to improve an agent's harness code — the code deciding what the model sees and which tools it calls — purely from run results, adapting agents to new tasks without manual tuning.
- The editor reads the harness plus failure report and writes a code change, rewarded by the new harness's score.
- Across 21 unseen reasoning task types, a 4B editor lifts average edit score from 0.32 to 0.62, surpassing its 35B teacher.
- An editor trained on HotpotQA keeps improving harnesses on 2 other QA benchmarks.
- Tip: run multiple edit rounds per task; repeated edits beat single fixes.
Related event: Harness Learning: 4B Model Rewrites Agent Code, Beats 35B Teacher(3 posts)→
More from coding & agent
- ZXFLASH compiles ZX Spectrum BASIC straight to Flash SWF bytecode running in Ruffle — danbri · 2026-10-24
- Enclave opens autonomous security agent fleet to everyone, launches Threat Feed vulnerability feed — BenBajarin · 2026-10-24
- 1979 paper dismissed formal verification; 2026 forecast says it'll be as common as fuzzing — danbri · 2026-10-24
- Scale AI Labs builds high-fidelity airline RL simulation from real JFK and Boston flight data — menhguin · 2026-10-24
- Running agents side by side on two iPhones: analyst ditches the desk for research — BenBajarin · 2026-10-24
- Analyst finds dual-iPhone setup surprisingly great for managing AI agents — firstadopter · 2026-10-24