HEELO Harness Framework Lifts Terminal-Bench Resolution from 37.3% to 55.5%
Haibo Jin · hf · 2026-11-09
A new paper introduces agent-native primitives and HEELO, a harness engineering framework that tackles the 'last mile' problem: models can reach resources but fail to apply them.
- Reusable primitives each carry a resident LLM that applies a resource in context and communicates with other primitives; HEELO composes task-specific graphs, verifies against environment evidence, and revises implicated primitives on failure.
- Results: Terminal-Bench 4.0 resolution rises from 37.3% (Codex) to 55.5%; Terminal-Bench 2.0 Pass@1 from 60.0% to 72.2%; ToolBench from 55.8% to 75.1%; GAIA-Text-103 from 45.6% to 58.3%.
- On 46 real-world tasks at heelo.dev, Claude Code and Codex each complete 9 alone but 39 and 35 with HEELO via MCP, with 1.9-2.3x less time and 1.55x lower cost per completed task.
- Figures show HEELO leads all 26 benchmark comparisons with efficiency gains in every primitive family.
More from coding & agent
- Nous Research launches Hermes plugin hackathon with $20k prizes; conductor plugin orchestrates coding agents — Teknium · 2026-11-09
- Cloudflare launches WebMCP developer preview: any site can serve browser agents with zero code — tristanbob · 2026-11-09
- Simon Willison: Claude Opus 5.5 Ports Three.js 3D Scene to Vanilla JS + WebGL — Simon Willison · 2026-11-09
- Simon Willison hides AI PR descriptions in details/summary tags — Simon Willison · 2026-11-09
- Team's AI agent fixes 40 of 41 user-reported issues in a single weekend — julianweisser · 2026-11-09
- NVIDIA's AgentBaton: Well-Timed Strong-to-Weak Model Handoffs Slash Agent Costs — nvidia · 2026-11-09