Google paper: gated modular harness edits match full rewrites at ~1/10 the token cost
rohanpaul_ai · x · 2026-11-20
A new Google paper finds self-improving agent harnesses work best with gated modular edits: an optimizer edits one of four modules at a time, and each change must pass a validation set and token cap.
On 40 held-out SWE-bench Lite tasks with Gemini 3.8 Flash, the gated harness solved 40.0% (vs 5.0% seed) at 46.7k tokens per task. An unchecked full rewrite hit 42.5% but burned 451.6k tokens per task — nearly 10x more — while prompt-only rules collapsed accuracy to 2.5%.
Takeaway: gate each harness change on held-out tasks and token cost, and enforce rules in code, not prompts.
Related event: Google Paper: Gated Rewrites of Agent Harnesses Cut Token Cost 10x(3 posts)→
More from coding & agent
- Apple reportedly building proactive AI agent that acts autonomously on users' behalf — burny_tech · 2026-11-20
- NormGym benchmark shows agents learn from own norm violations but not from others — xuanalogue · 2026-11-20
- MAC Bench: 8-agent teams beat solo agents via headcount, not coordination — gregd_nlp · 2026-11-20
- Every adopted 'compound engineering' company-wide so its codebase gets easier to maintain — every · 2026-11-20
- Not every agent task needs a general LLM — specialized small models by task type — ziv_ravid · 2026-11-20
- Start custom LLMs at 30B: runs on a single GPU and works with LoRA — ziv_ravid · 2026-11-20