ControlScope: Workflow Revision and Reliability in LLM Agents
Jingjie Ning, Xueqi Li, Yibo Kong, Dongting Li
cs.AI, cs.CL, cs.SE
2026-09-28
KEEP/ARG/FULL scopes tested from one state: full rewrite gains 2-3 of 20 tasks with reasoning, drops below baseline when fast, and one panel burns 29.7x tokens for +2 tasks.
Agents in the CodeAct style generate an executable program and let the program drive the tools. Once that program is running, how much of it should the model still be allowed to change? Systems differ in practice: let it run to completion (KEEP), allow edits to the data arguments of the next tool call, say a file path (ARG), or allow replacing the entire remaining workflow, for example turning per-file reads into one batched read (FULL). Prior work such as GraSP compared local repair with global replanning, and reflection-style methods revise between trials, but no one held the execution record and environment state fixed and treated revision permission itself as the controlled variable. This paper fills that gap: how much adaptation does wider revision scope actually buy, and at what cost.
ControlScope defines three nested operation sets. Every branch starts from the same (h, s): h is the full public execution record (goal, observations, current program, pending call) and s is the environment state, with model, APIs, and permissions held constant. FULL contains ARG, which contains KEEP, so any FULL loss traces to what the model actually chose, never to a missing permission.
| Setting | KEEP | ARG | FULL |
| 20 filesystem tasks, reasoning reviewer, 3 draws | 13 | 13-14 | 15-16 |
| 20 filesystem tasks, fast reviewer, 4 draws | 13 | 11-13 | 10-13 |
| ALFWorld, 87 tasks (fast and reasoning identical) | 85 | 86 | 87 |
| ALFWorld, 134 tasks across four scenes | 134 | 134 | 127 |
| AppWorld 585 instances, Normal split | 144 | 141 | 145 |
The direction flips with configuration. Under reasoning reviewers FULL reliably solves 2-3 more tasks; under fast draws the best case is a tie and the worst is three fewer. The two ALFWorld cohorts point opposite ways: +2 for FULL on 87 tasks, seven losses on 134. On AppWorld, FULL adds one task per split, paired gains of +0.60 and +0.24 percentage points with bootstrap intervals crossing zero, and the setup favors FULL there because each rewrite renews a 250-request block allowance.
On the 87-task cohort, FULL buys two extra successes over KEEP for 29.7 times its output tokens. It logs 34.5% less output than ARG, but 98.3% of that saving sits in two heating tasks where ARG produced 28 invalid reviews.
Two diagnostics carry the most information. At a single early boundary, all 216 paired draws end identically for ARG and FULL; of 432 decisions, 391 continued unchanged, with 18 argument patches and 23 workflow replacements. Frozen replays of two failed file-classification runs show that 49 of 71 and 46 of 62 chosen replacements would have reached the goal if retained; a failed file-organization run made 35 replacements and exhausted its budget, yet keeping only the first replacement and handing off to ordinary planning finished in 77 calls. Working repairs existed inside the policy's own outputs. Later reviews interrupted them.
Three smaller results: a midpoint schedule gives sustained review 14/20 versus 13/20 for one-time, at 294 reviews versus 20; a five-call protection rule saves 19.4% of logged output and loses 1 of 20 tasks; and in one shortcut case ARG widened a read-path list from 2 to 75 entries and finished in 11 calls while the sampled FULL skipped that cheaper route and took 25.
For agent builders the recommendation is concrete: expose both interfaces. Batching and parser repairs require workflow replacement, while one argument edit saved 14 calls. Review timing and valid-review rates belong in evaluation next to success rates; on the same panel ARG logged 51 invalid reviews against 16 for FULL, so the interface itself changes model behavior.
The deeper point is methodological. Granted scope is not realized scope: a bigger action set does not make the model pick the better operation, and a picked operation is not a completed one. The assumption that more autonomy yields more adaptation fails outright in the fast draws and the 134-task cohort. Read this as a measurement paper rather than an improvement; it takes one architectural choice apart and reports both directions and the price list.
The authors state the main ones themselves: 20 filesystem tasks across seven categories, with bootstrap intervals wide enough that one fast panel sits at [-30.0, -4.2] percentage points; interruption evidence stays at case level; conclusions depend on the source program, and swapping it can reverse the direction. A single model line, deepseek-flash, runs throughout. AppWorld's allowance renewal favors FULL, so the true difference may be smaller than reported.
Two things look shaky on a close read. Invalid-review counts vary by condition, so the comparison mixes revision scope with interface-induced failures to return a valid operation. And the opposite ALFWorld directions are attributed to cohort composition without a mechanistic account. The paper also discloses that generative AI assisted from question formulation through writing, with outputs checked against saved execution records.