RobustSGPO lifts agent harness completion from 60% to 80% on held-out tasks
dair_ai · x · 2026-09-10
dair-ai flags RobustSGPO for anyone running automated prompt or harness evolution. Semantic-gradient prompt optimization improves an agent harness from execution feedback, but its local update rule never decides how large an edit to request or which operation to apply. RobustSGPO adds that: it specifies the requested edit, constructs and checks the patch before accepting, and continues search from either the incumbent or a retained snapshot.
Measured over 120 tasks, 95 runs, and 7,350 candidate attempts in the AgentX brainstorming workflow under a 20M-token budget: completion on 30 held-out tasks rises from 60.0% to 80.0%, and test quality from 3.77 to 4.14.
Notably, periodically scheduling edit permissions (1→2→3) beats a fixed maximum by 0.28 test-score points — how much the optimizer may change per step is itself worth scheduling. Snapshot retention also proves effective.
More from coding & agent
- Agent-built small classifier labels 191K documents for $0.70, vs $13–26 with batch LLMs — vanstriendaniel · 2026-09-11
- Microsoft MVP Demos Personal AGI: Five Tools, Markdown and One LLM — dfinke · 2026-09-11
- Maestro diagnoses AI coding workflow issues with 25 targeted commands — tom_doerr · 2026-09-11
- CMU Researchers Unveil TAHI, a Human-Agent Interaction Framework for Expert-Grade AI Output — EchoShao8899 · 2026-09-11
- From WebSocket error to merged PR: GitHub Copilot App handles the full loop — DanWahlin · 2026-09-11
- raggy: A Lightweight CLI Tool for Fully Local RAG Over Your Documents — paulknysh · 2026-09-10