CMU Study: Harness Engineering Boosts Scientific Agents Without Retraining
niloofar_mire · x · 2026-11-04
Researchers at CMU's Looni Lab (Suresh Raghu, Kevin Han, Aviral Kumar, Niloofar Mireshghallah) examine whether human taste is overrated in agent harness engineering.
- Harness engineering can substantially improve agents without retraining the underlying model, and parts of the design process are increasingly automated.
- Coding agents often reuse familiar harness components, but scientific agents typically need interfaces built around a specific domain and its tools.
- On SMDD-Bench (drug-design tasks: lead optimization, scaffold hopping, interaction points), automated search successfully refined workflows and discovered useful scientific heuristics, while the strongest manual improvements came from changing what evidence the model could observe and what state the environment preserved.
- Takeaway: the answer is task-dependent — automation and human-driven representation/state design each win in different places.
More from coding & agent
- ThunderAgent merged into LLM-D: 2x agent serving throughput, TTFT from ~50s to 2-4s — simran_s_arora · 2026-11-04
- Agents pass the test but fail the job: the requirement gap and the model gap — bigdata · 2026-11-04
- Does that new prompting trick actually work, or just look better? — jimmykoppel · 2026-11-04
- Vercel launches Security Review in public beta, free until Nov 10 — cramforce · 2026-11-04
- QuickProof: formal verification for C with specs written as plain C tests — thomasahle · 2026-11-04
- Project Moah: Open-Source Memory Linker That Gives AI Agents a Brain — starcount_games · 2026-11-04