PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents
Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Wen Da, Zhaokai Luo, Mu Chuan, Yao Hu, Wenjie Li, Chengyue Jiang
cs.AI
2026-08-27
PILOT steers workers mid-run and distils skills live, reaching 71.9% one-shot on Terminal-Bench 2.0 and +14.6 points over 20 iterations while cutting output tokens ~43%.
Long-horizon agent runs produce experience that could help the current attempt and later ones. Successful traces expose procedures worth keeping as skills. Failed traces expose strategies that should not be repeated. Most self-improvement pipelines wait until the run is over. Reflexion, judge-based scoring, and self-evolving harnesses all improve the next episode. They cannot recover the episode that generated the lesson, and they cannot test a newly extracted skill against the same execution that revealed it.
The architecture is also split in the wrong place. Single-agent self-correction is timely, but execution debris and diagnosis share one context window, so a bad strategy is hard to notice while the agent is still buried in tool output. Subagent delegation isolates execution, yet the parent typically sees a summary only after the child returns. Live correction and a dedicated self-improvement role do not coexist in either design.
PILOT is a supervisor-worker harness built on the Pi coding-agent runtime. Model weights stay frozen. What evolves is the persistent harness: a skill library and memory. The same frozen backbone fills both roles, so gains cannot be explained by a stronger supervisor model.
Two loops share one stream of execution experience.
Worker contexts absorb tool traces and dead ends. The supervisor context stays on the goal, recent events, and signs of drift. Trajectory fragments are inspected only when diagnosis is needed. Several workers can run in one episode.
Two open frozen backbones, GLM-5.1 and Kimi-K2.6, and three long-horizon benchmarks. Each configuration is run twice and averaged. Thinking is on, generation cap 32k. In the one-shot setting every task starts from a fresh harness, so the comparison isolates live steering.
Terminal-Bench 2.0 (89 tasks), overall pass rate:
| Harness | GLM-5.1 | Kimi-K2.6 | Avg |
| Terminus-2 | 64.0 | 59.6 | 61.8 |
| Hermes | 60.1 | 64.0 | 62.1 |
| OpenCode | 66.9 | 64.6 | 65.8 |
| Pi | 65.7 | 66.9 | 66.3 |
| PILOT | 71.9 | 71.3 | 71.6 |
On Hard tasks PILOT scores 55.0% with both backbones, 5.0 and 6.7 points above Pi (50.0% / 48.3%). On SWE-bench Pro the average is 59.9% versus Pi at 55.5%; with Kimi-K2.6 the gap is 65.1% vs 59.1%. PILOT ranks first in five of six backbone-benchmark pairs. The miss is SWE-bench Multilingual with Kimi-K2.6: 73.7% against Pi's 75.9%.
The self-improvement setting sweeps Terminal-Bench 2.0 for 20 iterations. During a run the harness sees only the live trajectory and environment feedback, never verifier scores. After the sweep, updates from successful runs are merged into the next shared state; failed-run updates are dropped. Every harness gets the same retain-and-reuse instruction and the same initial skill library. PILOT's best observed pass rate rises from 66.3% to 80.9% with GLM-5.1 (+14.6, first peak at iteration 14) and from 68.5% to 80.9% with Kimi-K2.6 (+12.4, first peak at iteration 13). OpenCode gains 7.9 points; Pi gains 2.3. The skill set grows 62 to 83 and 50 to 81. Mean output tokens per evaluated task fall from 28.5K to 16.3K (-42.9%) and from 41.9K to 22.1K (-47.4%). Successful evaluations per million output tokens rise 110.3% and 134.0%. Extra passes concentrate on Hard: +8 tasks for GLM, +12 for Kimi.
A manual audit of full supervisor-worker traces counts a success as steering-aided only if the supervisor names a concrete error or stalled strategy, the worker follows that correction, and the corrected path is the one that passes. Easy: 0%. Hard: 6.1% (GLM) and 19.7% (Kimi). Overall: 2.3% and 10.6%. Two traces: a CoreWars worker spent twenty minutes tuning DAT-clear variants against synthetic opponents until the supervisor told it to fetch a published warrior; a tensor-parallelism worker added bias before allreduce, which would scale bias by worldsize, until the supervisor caught it.
This is a harness-level incremental gain, not a new model. Weights never move. Correction of the active run and accumulation across related tasks sit in one loop. For people building long-horizon coding or terminal agents, the practical recipe is an independent supervisor session on a Pi-style runtime, closer to how teams already ask an agent to keep skills between related jobs than a post-hoc Reflexion pass.
The numbers also mark the ceiling. One-shot is already about 5 points above Pi. The large gap appears when the same related task set is repeated. Hard tasks benefit most from reusable procedures. Token drop means later iterations spend less on rediscovery; the extra score is not bought with extra generation.
The paper states two limits. Twenty-iteration sweeps are expensive, so the study uses three benchmarks and two open-weight backbones; proprietary models are untested. Supervisor and worker share a backbone, so quality, cost, and oversight trade-offs of mixed pairings are unknown.
A few other issues sit in the protocol. Writes happen during execution, but cross-iteration retention still filters on verifier pass or fail. That is earlier than a full post-hoc summary, and it is still not unsupervised live evolution. One-shot GLM at 71.9% and iteration 0 at 66.3% are not comparable: the two settings differ in instructions and initial skills. Steering-aided successes are only 2.3% and 10.6% overall, rising on Hard, so a large share of the one-shot lift may come from role separation and extra context rather than mid-run redirects. SWE-bench drops 43 Multilingual and 198 Pro tasks over JavaScript runtime conflicts; the cut is applied to every harness, but absolute rates are not comparable to the public leaderboard. PILOT extends Pi, so the Pi baseline is the cleanest control; other harnesses do not share the same runtime substrate.