RobustSGPO lifts agent harness completion from 60% to 80% on held-out tasks
dair_ai · x · 2026-09-10
dair-ai flags RobustSGPO for anyone running automated prompt or harness evolution. Semantic-gradient prompt optimization improves an agent harness from execution feedback, but its local update rule never decides how large an edit to request or which operation to apply. RobustSGPO adds that: it specifies the requested edit, constructs and checks the patch before accepting, and continues search from either the incumbent or a retained snapshot.
Measured over 120 tasks, 95 runs, and 7,350 candidate attempts in the AgentX brainstorming workflow under a 20M-token budget: completion on 30 held-out tasks rises from 60.0% to 80.0%, and test quality from 3.77 to 4.14.
Notably, periodically scheduling edit permissions (1→2→3) beats a fixed maximum by 0.28 test-score points — how much the optimizer may change per step is itself worth scheduling. Snapshot retention also proves effective.
More from coding & agent
- Spring AI 2.1.0-RC1 adds tool-call confirmation, pausing agent loops before real-data access — therealdanvega · 2026-10-26
- 250 Builders Hack With Codex at Berlin Community Hackathon Ahead of OpenAI DevDay — paw_lean · 2026-10-26
- Building AI bots on raw APIs? Devs say models are 'dumb without the harness' — madeofAjala · 2026-10-26
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- Microsoft demos intent-classifying agents built with Copilot App and Foundry, deployable across Teams and Outlook — marlene_zw · 2026-10-26
- Looking for a wearable mic with best battery life that feeds recordings to personal agents via MCP — altryne · 2026-10-26