GUI-HARVEST auto-optimizes agent harnesses; frozen GPT-5 gains 13.87 points
CUHK-SZ · hf · 2026-10-07
CUHK-Shenzhen released GUI-HARVEST, an automatic harness optimizer that self-improves GUI agents with frozen backbones—optimizing how observations are assembled, actions executed, and verification/recovery/termination controlled. Three components:
- Aligns model outputs and executed actions with before/after screenshots, grounding diagnosis in specific UI transitions;
- Treats repeated runs of the same task as a joint evidence unit, using within-task comparisons to locate outcome-relevant behavioral differences;
- Consolidates verified findings into recurring failure patterns, mapping them to bounded source-code edits with pre-recorded predictions checked via repeated execution.
On OSWorld-Verified, six backbones show consistent held-out gains (Qwen3-VL-32B +12.33 on the full suite); frozen-harness transfer lifts GPT-5 by 13.87 points on WindowsAgentArena at 50 steps.
More from coding & agent
- Survey maps in-parameter memory methods for LLMs by placement and acquisition time — _reachsumit · 2026-10-07
- Ill-specified real work means harnesses still matter, argues Paras Chopra — paraschopra · 2026-10-07
- Agentic AutoRAG: LLM Agents Diagnose Retrieval vs Generation Failures to Tune RAG Pipelines — _reachsumit · 2026-10-07
- Cursor adds Cloud Agents API endpoints for environment builds with status and error codes — tetsuoai · 2026-10-07
- Figure CEO: filling Vietnam's brutal visa form was our AGI test — now an agent passed it — adcock_brett · 2026-10-07
- Vite+ 1.1 released: 24% faster vp dev startup, 30% less memory, clearer prompts — irvinebroque · 2026-10-07