Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems
Shuang Yang, Zijie Zhuang, Changxin Lao, Pengbo Xu, Hanwen Xu, Yusheng Huang, Han Gao, Guanchen Wang, Tianbao Ma, Linxun Chen, Peilin Song, Xuming Wang, Chen Li, Fan Wu, Tao Wang, Zibo Zhao, Xiangyu Wu, An Liu, Fei Pan, Peng Jiang, Chen Yang, Zhaojie Liu, Wenwu Ou
cs.AI, cs.IR
2026-09-24
Research and Model agents ran 636 recsys experiments in ~25 days; 560 beat business-baseline AUC. Five A/B tests: 10-15% acquisition, 15-20% target ad spend, 0.3-0.8% watch time.
Industrial recommendation research is a sequence of experiments. An engineer reads a result and then chooses to refine the model, chase an anomaly, or switch direction. Agents can already write code and revise architectures. What they usually cannot do is decide which implementation, measurements, and open questions should start the next experiment. AgentX-Model puts that loop inside a sandbox: business inputs and prediction tasks stay fixed, how the model uses those inputs may change, and deployment gates stay human.
Earlier AgentX work generated paper-driven proposals and composed methods across papers. This report feeds experimental evidence back into later proposals, so a result becomes a starting point rather than a one-off score.
Two agents split the horizon. The Research Agent reads papers, experiment records, and business feedback, then writes a proposal with a research question, a starting implementation, a change, and an evaluation design. An Auditor reviews it in a separate context. The Model Agent implements the approved proposal over multiple rounds of coding, training, and measurement. Better intermediate checkpoints are kept even when later rounds regress. The language model is privately hosted GLM-5.2. The Model Agent runs on Claude Code and submits jobs to an internal KML platform.
Continuing research is typed into four actions:
A scheduler rotates action types. Inside the chosen type, the Research Agent builds a concrete experiment from available evidence, or defers. Comparisons use three references: the business baseline, the direct parent, and the strongest comparable ancestor. Beating a parent without beating the path best is a local recovery, not a new record.
Over about 25 days, 560 of 636 completed model-changing experiments recorded AUC above their business baselines. Diagnose runs are excluded from that gain table.
| Setting | Baseline AUC | Best AUC | Above baseline / valid |
| A | 0.773683 | 0.797352 | 500/527 (94.9%) |
| B | 0.815403 | 0.823145 | 21/29 (72.4%) |
| C | 0.816849 | 0.820246 | 20/33 (60.6%) |
| D | 0.809656 | 0.811432 | 19/47 (40.4%) |
Those shares are not success rates. Many Follow-ups and Compositions inherit implementations that already beat the baseline. In Scenario A, 7 of 77 Follow-ups with complete ancestry beat every ancestor, and 5 of 120 Compositions did. Among 189 multi-round experiments, 58.7% improved after the first valid round, and 48.1% ended below an earlier best. The best measured code has to be retained with its evaluation context.
Five recent Launch Review A/B tests report 10-15% acquisition efficiency, 15-20% and 5-10% target-segment ad spend, 0.3-0.8% watch time with about 10% fewer FLOPs and parameters, and 0.5-1% daily active users. On a Scenario A calibration path, a multiplicative correction estimated from training statistics cut test |PCOC-1| error by about 22.8% while keeping AUC above the business baseline. In a 10-day sample, 886 proposal attempts produced 262 deliveries, median 33.6 minutes.
A dependency-aware historical replay over 473 experiment nodes and six environments compares allocation policies. When agents already inspect candidates and pick a concrete experiment, more complex bandit or joint type-and-candidate selection does not consistently improve research efficiency. Fixed action rotation plus agent candidate selection remains a strong baseline.
This is an operations report on long-horizon agent research inside industrial recommenders, not another one-shot AutoML score. The transferable piece is the action set plus the rule that round-level code and measurements survive later regressions. Teams that already have a training platform and a business baseline can copy the propose, investigate, and evidence-return loop without copying any particular network.
The sandbox forbids changing upstream features or prediction tasks, so agents cannot invent new business problems. Above-baseline rates are inflated by inherited wins; path-best rates are the harder bar, and they are 9.1% for Follow-up and 4.2% for Composition in Scenario A. Online numbers are coarse ranges from five shipped Launch Reviews, not from all 636 runs. In Scenario D, paper-derived Reproduce is 0/7 above baseline versus 5/15 for cross-setting transfer, with small unmatched cohorts. Replay can only return outcomes for combinations that were actually run. Model weights stay frozen; evolution is in tools, instructions, and memories, all human-approved.