Tsinghua & Microsoft's HarnessMM cuts agent token use by up to 87.93% while raising success rate
rohanpaul_ai · x · 2026-11-02
A Tsinghua and Microsoft paper, HarnessMM: Recursive Self-Improvement for Harness Token Efficiency, tackles a hidden inefficiency in agent harnesses: LLMs are re-invoked after every tool result even when the next step was already decided.
How it works: the model pre-writes future commands with exact arguments plus pass checks (e.g. a successful build). When a tool result passes the check, the agent executes the command directly and skips the LLM call; novel semantic decisions re-invoke the original model.
Results on 98 HumanEval tasks:
- Codex: success held at 90.82% while tokens/task dropped from 66.72 to 47.02 (29.54% saving)
- DeepSeek Harness: success rose from 71.43% to 85.71% while tokens/task fell from 96.28 to 11.62 (87.93% saving)
- MBPP ablation: 26.91% cheaper than single-use graphs, 19.52% cheaper than retained callbacks
Practical takeaway: if your agent runs predictable chains like write → build → test, have the model plan them once with pass checks instead of re-asking the LLM each step.
More from coding & agent
- Microsoft's MemSteer turns unlabeled user histories into agent memory supervision — MicrosoftResearch · 2026-11-02
- She fired herself from her own Instagram: AI agent runs the account, views up 82.3% — heyshrutimishra · 2026-11-02
- Sourcegraph: AI writes code faster than teams can review it, driving more outages — AI Engineer · 2026-11-02
- GitHub launches Agent HQ: Anthropic, OpenAI, Google coding agents in Copilot — emmanuelvivier · 2026-11-02
- DeepSeek open-sources Super-Agent, a coding agent that reads team chats to minimize human input — emmanuelvivier · 2026-11-02
- JPMorgan, Schwab and Amazon block AI shopping agents; Schwab says it stops dozens of attempts weekly — emmanuelvivier · 2026-11-02