Zoom study of 176 settings: context management saves tight-window coding agents, planning just cuts cost for strong models
ZoomCommunications · hf · 2026-09-18
A Zoom team study on Hugging Face examines coding-agent harness design by holding the execution loop fixed and varying three components — planning, action space, and context management — across 4 models on SWE-Bench Verified and Terminal-Bench 2.1, spanning 176 matched settings (5 context-management strategies, 4 context-window budgets, and targeted ablations).
Key findings:
- Context management matters most as the context-window budget tightens, with most benefit coming from preventing context-overflow failures.
- Staging rule-based elision before LLM-based summarization is the most efficient strategy; making elided content recoverable adds machinery models rarely use, with no accuracy gain.
- Planning shifts from an accuracy scaffold for weaker models to a pure cost saver for stronger ones, with little accuracy change.
- Action space: predefined tools help models with weaker bash proficiency; bash-capable models work well with a bash-only interface at substantially lower cost, especially on CLI-centric tasks.
- Trajectory analysis: context management extends trajectories without changing behavior, planning changes where trajectories stop, and action space changes code-writing granularity.
The work offers a modular framework for model- and budget-aware harness design.
More from coding & agent
- WebMCP draws developer buzz as a way to bring MCP into the browser — nickbaumann_ · 2026-09-18
- Demo: Claude with built-in Cowork and Artifacts builds a multiplayer game from one prompt — felixrieseberg · 2026-09-18
- Six Coding Agents Share the Same Plan Mode Shape but Split on Context After Approval — RunAI_Coder · 2026-09-18
- EvoSkill-GUI: training-free skill evolution lifts GUI agents by up to 16.2% — Bofan Chen · 2026-09-18
- Testing Non-Deterministic LLM Agent Pipelines with Property-Based Assertions — Repulsive_Sugar_5252 · 2026-09-18
- Developer: Listing Rules for AI Never Ends — Describe How the Answer Should Feel — gabriel1 · 2026-09-18