LongHorizon-Harness: external task state + audit loop triples OSWorld agent scores
rohanpaul_ai · x · 2026-08-30
arXiv paper LongHorizon-Harness (2608.01964) tackles LLM agents on long-horizon tasks:
- Problem: existing harnesses keep execution, task state, and completion assessment inside a growing context, making state hard to track and letting wrong self-assessments propagate into later decisions.
- Method: reformulates long-horizon execution as task-state management — state is maintained explicitly outside execution and updated only with facts independently verified from the environment. A Manage-Execute-Audit (MEA) loop uses a manager to maintain state and pick subtasks, a fresh-context executor, and a read-only auditor verifying environment state each round. A lightweight AgentAdapter swaps models and harness backends without touching native agent loops.
- Results: Qwen 3.7-Plus improves from 51.8% to 80.7% on WeaveBench, 69.7% to 77.2% on Terminal-Bench 2.1, and 2.8% to 8.3% on OSWorld 2.0; Claude Opus 4.7 rises from 20.0% to 34.3% on an OSWorld 2.0 subset — consistent gains across models, harnesses, and domains.
More from coding & agent
- Vercel Goes Full Backend: Python GA, Containers, and Daemons — cramforce · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Vercel Ship Demos Container Deploys for Go, Ruby, Java and More — edwin · 2026-10-16
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Databricks engineers land impactful PRs on day 2 with AI dev tools, vs week 3 before — Yuchenj_UW · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16