TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent
Jie Yang, Yan Zheng, Jiarui Sun, Xiran Fan, Junpeng Wang, Liang Wang, Zelin Xu, Qinghua Liu, Zhengyu Fang, Yiwei Cai, Philip S. Yu
cs.AI, cs.LG
2026-09-23
TimeEvo grows evidence-only tools from failures and admits them via a paired fix-vs-harm gate. From an empty library it lifts all 30 task-backbone runs, mean +7.2 to +8.8 pp.
Time-series agents answer by calling tools that people install before the agent runs. A Visa Research and UIC team froze the LLM, attached TimeART's 21 expert tools, and paired every question across ten QA tasks and three backbones. The catalog humans prefer to ship is not the catalog the agent benefits from.
Gains and losses keep the same sign on all three models. The library includes a dedicated anomaly detector, yet anomaly accuracy still falls 5.8 to 8.8 points on every backbone. T3 shares a family with T2 and T4, which improve; T3 degrades on all three models. Tools are issued once per task. Whether a tool helps is decided question by question at runtime.
Generic self-revision hides the damage. One Self-Refine pass rewrites 147 answers across the ten tasks: 49 fixes, 56 previously correct answers broken, 42 wrong-to-wrong swaps. The score moves by less than a point. Two other backbones rewrite 97 and 163 answers, break 45 and 56, and again never move a full point. On T4, luna changes 14 answers, fixes 0, breaks 11. Harm cancels repair inside one average.
TimeEvo starts from an empty library and treats training errors as the spec for new tools. An update ships only if it repairs more than it breaks.
At inference the frozen model answers first. Admitted tools review only in-scope questions and keep the incoming answer by default. Tool errors, no-ops, and indecisive reviews fall back. Out-of-scope items cannot be degraded.
TimeEvo is the only method that improves every task on every backbone, 30 of 30. Mean gains: +8.79 on GPT-5.6-luna, +8.73 on GPT-5.4-mini, +7.21 on GPT-5.6-terra. Each backbone grows its own library from its own failures.
| Method | Ten-task mean Δ | Anomaly Δ |
| TimeEvo / luna | +8.79 | +14.69 |
| TimeART 21 tools | mixed | −6.25 / −8.75 / −5.79 |
| Fixed Chronos-2 | strong on T2/T4 only, up to +22.22 | flat or down elsewhere |
| Self-Refine | score moves < 1 pp | luna: 147 changed, 56 broken |
| vote@5 | +0.79 / −0.21 / +1.64 | well below TimeEvo |
Few-shot ICL reads the same training split and goes clearly negative on Classification under all three backbones.
Install luna's library into Claude-opus-5, Claude-sonnet-5, and GPT-5.6-sol with no test-set selection. Bare ten-task means sit between 54.6 and 57.1, no better than the cheap model. The same library then adds +10.50, +10.65, and +8.27. One of thirty transfer cells is negative: Match on sol at −2.50. On Anomaly the two Claude models gain +26.25 and +26.50, above luna's own +14.69. The numbers the tools compute do not change; stronger models read them better. Growing tools on an expensive model is unnecessary.
Ablations sit on Anomaly and TSAQA-DT, mini and terra, four cells chosen in advance. Drop train-only calibration trees and both Anomaly runs deploy nothing. Drop the gate and the mini Anomaly library finishes 1.25 below its own base; the full method finishes 16.25 above. Drop structured scopes and accuracy falls 1.00 to 9.25 points. Unconditional overwrite of the same library fixes 16 to 25 extra answers per cell and breaks far more: 59 vs 19, 32 vs 22, 26 vs 3, 28 vs 10. Starting from Chronos or from the 21 expert tools underperforms an empty root in all four cells.
Across 30 one-round runs, 4,910 training errors yield 113 deployed artifacts. Twenty-two rounds pass outright, seven pass after one prune, one rolls back. The gate has one false positive, a T1 library at −1.27 test, and one conservative miss, a Merrill library whose diagnostic scored +5.00.
Tooling a time-series agent can be a measured loop rather than a pre-deployment guess: failures write the missing measurements, and a fix-versus-harm gate decides what ships. Under closed-set QA with labeled train and validation splits, all 30 cells move up, by about eight points on average.
For anyone already running a time-series QA agent, the gate is the portable piece. A score that barely moves can still flip 56 correct answers. Self-Refine, five-sample voting, and expert catalogs all hide that breakage. A library grown on luna still lifts stronger models, so the missing ingredient is measurement, not a bigger LLM.
This is incremental work on closed-set, labeled QA. It is not a drop-in for open-ended analysis pipelines.
The paper has no standalone Limitations section.
Evolution needs gold labels on train and validation. Failures have to be auto-gradable, and the gate has to count helped versus harmed on the same questions. Unlabeled production logs cannot drive this loop. Main results are frozen to a single round.
Every task is multiple-choice time-series QA. Series lengths run from 8 to 1460 points and option counts from two to six, so raw accuracies are not comparable across tasks. The paper does not test open-ended analysis, multi-turn dialogue, or production workflows. Anomaly series have median length 16.
The gate errs in both directions: it accepted a test-negative T1 library and rejected a Merrill library with a +5.00 diagnostic. Tools may call Chronos-2, and Anomaly libraries carry train-only trees, so synthesis from failures is not invention from scratch. Reproduction is tied to GPT-5.6 / GPT-5.4 family APIs.