LongHorizon-Harness: external task state + audit loop triples OSWorld agent scores

rohanpaul_ai · x · 2026-08-30

arXiv paper LongHorizon-Harness (2608.01964) tackles LLM agents on long-horizon tasks:

Problem: existing harnesses keep execution, task state, and completion assessment inside a growing context, making state hard to track and letting wrong self-assessments propagate into later decisions.
Method: reformulates long-horizon execution as task-state management — state is maintained explicitly outside execution and updated only with facts independently verified from the environment. A Manage-Execute-Audit (MEA) loop uses a manager to maintain state and pick subtasks, a fresh-context executor, and a read-only auditor verifying environment state each round. A lightweight AgentAdapter swaps models and harness backends without touching native agent loops.
Results: Qwen 3.7-Plus improves from 51.8% to 80.7% on WeaveBench, 69.7% to 77.2% on Terminal-Bench 2.1, and 2.8% to 8.3% on OSWorld 2.0; Claude Opus 4.7 rises from 20.0% to 34.3% on an OSWorld 2.0 subset — consistent gains across models, harnesses, and domains.

More from coding & agent