Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
Ruiqing Yue, Yu Cui, Zhuoyu Sun, Sicheng Pan, Xianhong Xue, Tingyu Li, Ting Li, Wenzhuo Zhu, Yi Chen, Yifei Liu, Baohan Huang, Zhe Cui, Haibin Zhang, Cong Zuo
cs.SE, cs.AI
2026-09-10
Ecdysis batches cross-task failures, then multi-role diagnosis edits a frozen-model harness. Versus per-instance self-evolution it trains 1.84× faster and lifts accuracy 18.56%.
A runtime harness sits outside a frozen model at inference time: planning, tool use, context management. On six Qwen3 model-domain pairs, stripping every harness layer leaves 29.72% average accuracy. A fixed human-configured harness reaches 50.28%. Serial edits by a coding agent after each failure only get to 43.33%, worse than the human setup.
The hard part is attribution. A failed trajectory may come from the current model's blind spots or from a systematic hole in the interaction code. Model quirks want accommodation; harness holes want repair. Existing self-evolution treats every failure as an independent patch request. That burns wall-clock on repeated executions and code edits, and it tends to bake a one-off model quirk into a global constraint.
Ecdysis, named after insect molting, freezes the task model and edits only harness source, starting from the same human harness.
Each round has three stages. The current harness runs the training set. Trajectories scoring below threshold λ become failure records (task id, termination reason, tool-call history) and are grouped by pattern. Groups that cover at least two distinct tasks get priority; single-task groups stay as auxiliary evidence. A group is diagnostic, not a mandatory repair target.
Failure-Driven Collaborative Refinement (FDCR) then turns those groups into a modification spec. Analyst proposes a minimal change. Critic checks over-broad triggers, blocked legitimate actions, runtime-contract breaks, and regressions on previously solved tasks. Engineer tracks agreements and leftover disputes. They share one transcript for K passes. A Moderator writes the structured spec; OpenCode with DeepSeek-V4-Pro actually edits the code. The candidate is kept only if the overall training score strictly rises. Individual tasks are allowed to drop.
Whatever the failure count in a round, the coding agent is called once. That is the main reason training time falls. A failure-aware data rule prefers tasks that expose new execution paths; a quarter of the training failures (five items) nearly matches full-data training.
Harnesses evolve on Qwen3-8B for at most three candidate rounds, then freeze. Transfer tests use Qwen3-14B/32B, MiniMax-M2.7 (230B), and Llama-3.1-8B. Each of τ²-Airline and τ²-Retail has 20 train and 20 test tasks, three independent trials per test task.
Five models × two τ² subsets:
| Method | Avg. accuracy | Pass@3 | Pass^3 |
| Direct | 38.17% | 53.50 | 22.50 |
| Human-Aug. | 51.67% | 66.00 | 37.00 |
| Self-Evolution | 46.67% | 63.50 | 29.00 |
| Ecdysis (no FDCR) | 54.67% | 66.50 | 42.00 |
| Ecdysis (with FDCR) | 59.33% | 71.50 | 45.00 |
Self-Evolution lands below the frozen human harness. Round-level aggregation alone moves accuracy from 46.67% to 54.67%; FDCR adds another step to 59.33%. That is +27.1% relative to Self-Evolution and +14.8% relative to Human-Aug. Pass^3 rises 55.2% relative to Self-Evolution.
Adding AgentBench, mean accuracy across three benchmarks goes from 58.67% to 69.56% (+18.56% relative). On τ²-Airline, Qwen3-8B moves from 35.00% to 60.00%; Qwen3-32B, which never evolved the harness, moves from 51.67% to 68.33%.
Wall-clock on τ²-Airline: Self-Evolution 8,120.6 s, Ecdysis with FDCR 4,403.0 s (1.84×), without FDCR 2,510.8 s (3.23×). Most API spend is evolution, not evaluation: on Retail, $8.484 vs $2.485 vs $5.763. Held-out tokens fall from 11.57M to 10.16M (12.19%). Manual coding of the accommodation ratio t drops from 60.0% to 45.5%. With one-quarter of the failures, Ecdysis reaches 71.67% on Qwen3-32B τ²-Retail against 75.00% full-data and 65.00% random-five.
For people who ship agent runtimes, a failure log is not a patch ticket. Self-Evolution took a 51.67% human harness down to 46.67% on τ². Patching every miss can shrink the legal action space. Ecdysis raises the edit grain from instance to round and treats cross-task recurrence as an inductive bias for systematic defects.
What transfers is the procedure, not a drop-in harness. The base is Life-Harness; the editor is DeepSeek-V4-Pro. Aggregation buys most of the speed and some of the accuracy; FDCR spends extra evolution budget for quality. The code is public.
There is no standalone Limitations section. Each subset has 20 training tasks. Evolution runs only on Qwen3-8B, temperature 0.0, at most three rounds. t comes from manual reading of edit decisions, not an automatic metric. The authors state that cross-task recurrence is an inductive bias, not causal proof.
The numbers are uneven. On MiniMax-M2.7 τ²-Retail, the no-FDCR variant hits 100.00% and FDCR falls back to 96.67%. Llama-3.1-8B stays weak on Retail, 11.67% at best. AgentBench is near ceiling: Self-Evolution already at 90.00% on Qwen3-8B. The headline 18.56% is versus a Self-Evolution run that already lost to the human harness; the harder baseline is Human-Aug., +14.8% relative on τ². Latency is not uniformly better: Llama on Airline goes from 43.38 s to 159.60 s.