NVIDIA's SoL-Pi lets AI rewrite agent harnesses, cutting tokens 44.7-49%
alex_verem · x · 2026-09-23
A paper from NVIDIA, NTU and MIT (SoL-Pi, arXiv:2609.20519) hands harness tuning — the software layer around coding agents that reads files, runs commands and manages memory — to an AI instead of humans.
- The research AI proposed and tested harness changes across 150 research directions, 500 environments, 3,000+ runs and 60,000+ agent-environment interactions, keeping only token-saving fixes that didn't hurt results.
- Four surviving mechanisms: merging file edit + test run into one call; compressing memory when a subtask ends only if cheaper; replacing repeated large tool outputs with a 1KB excerpt plus on-demand pull; delegating log summarization to a cheap model with a verifier rejecting evidence-dropping summaries.
- On the 51-task EdgeBench, token traffic fell 44.7–49.0% and API cost by 1/3 vs Pi; it runs 50% cheaper than Codex's native harness and 54.3% cheaper than Claude Code's, holding 94% of baseline scores. Built on GPT-5.6 Sol, it transferred unchanged to Opus 5 saving 44.7% tokens.
- Takeaway: much of the waste sits in software around the model, and AI finds it faster than humans; the team's next step is rerunning the search with the cheaper harness.
Related event: NVIDIA's SoL-Pi auto-optimizes agent harnesses, cutting tokens ~45-49%(2 posts)→
More from coding & agent
- Dev's 'software factory' burns 5-10B tokens a day with 32 parallel coding agents — BLUECOW009 · 2026-09-23
- Dev shares how he built app moderation with Jev and LLMs — amos_gyamfi · 2026-09-23
- Dev argues the real AI agent problem is unmeasurable requirements, not capability — algo_diver · 2026-09-23
- $/task beats token price: outputs are <5% of tokens in agentic coding — xeophon · 2026-09-23
- Rethinking multiplayer agent UX: shared artifacts may beat a single group-chat box — max__drake · 2026-09-23
- Dev: if your eval costs $20 for 100 runs, the problem set isn't hard enough — pvncher · 2026-09-23