Zoom study of 176 settings: context management saves tight-window coding agents, planning just cuts cost for strong models

ZoomCommunications · hf · 2026-09-18

A Zoom team study on Hugging Face examines coding-agent harness design by holding the execution loop fixed and varying three components — planning, action space, and context management — across 4 models on SWE-Bench Verified and Terminal-Bench 2.1, spanning 176 matched settings (5 context-management strategies, 4 context-window budgets, and targeted ablations).

Key findings:

Context management matters most as the context-window budget tightens, with most benefit coming from preventing context-overflow failures.
Staging rule-based elision before LLM-based summarization is the most efficient strategy; making elided content recoverable adds machinery models rarely use, with no accuracy gain.
Planning shifts from an accuracy scaffold for weaker models to a pure cost saver for stronger ones, with little accuracy change.
Action space: predefined tools help models with weaker bash proficiency; bash-capable models work well with a bash-only interface at substantially lower cost, especially on CLI-centric tasks.
Trajectory analysis: context management extends trajectories without changing behavior, planning changes where trajectories stop, and action space changes code-writing granularity.

The work offers a modular framework for model- and budget-aware harness design.

Original post →

Zoom study of 176 settings: context management saves tight-window coding agents, planning just cuts cost for strong models

More from coding & agent