Meta-Harness auto-searches harness code, gaining 4.7 points on IMO-level math
zainhas · x · 2026-09-14
A new arXiv paper, Meta-Harness (Chelsea Finn, Omar Khattab, et al.), introduces an outer-loop system that automatically optimizes the harness — the code deciding what info LLM apps store, retrieve, and present — using an agentic proposer that reads source code, scores, and execution traces of prior candidates via a filesystem.
Results: +7.7 points over a SOTA context-management system on online text classification with 4x fewer context tokens; a single discovered harness adds +4.7 points on 200 IMO-level problems averaged across five held-out models; discovered harnesses beat hand-engineered baselines on TerminalBench-2.
Poster zainhas also notes open-source agent framework goose is 'pretty good out of the box' and that harness selection hides lots of alpha.
More from coding & agent
- dhh on coding in the AI era: code is no longer the bottleneck, humans are — NERDDISCO · 2026-10-31
- pi-rs: Open Source Rust Port of the pi Coding Agent Ships pi-durable, No Node Needed — jasonkneen · 2026-10-31
- Open-source ComfyUI Motion Previs Studio brings 3D camera blocking to AI video — navarisun · 2026-10-31
- Replit agent drives Chrome via Computer Use in new demo, CEO Amjad Masad amplifies — amasad · 2026-10-31
- Every Used Codex and Agents to Survive the Railway Credential Breach — every · 2026-10-31
- Google's New /goto Anti-Bot Delay Hits SERP-Reselling Agent Search Firms — jobergum · 2026-10-31