RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Haoran Ling, Yuecheng Li, Zeyu Song, Jing Yao, Shuwen Kang, Chi Lu, Wenjin Wu, Peng Jiang
cs.IR, cs.AI, cs.CL
2026-07-31
RecHarness splits recommender tuning: a Thompson-sampling bandit picks the edit direction, an LLM writes the code. A 7-day online A/B lifted advertiser value 2.084%, revenue 0.534%.
Tuning a modern recommender model is still mostly manual: change the architecture, the loss, the training strategy, see whether the validation metric moves, try again. LLM agents can automate this trial-and-error, but if the same LLM both picks what to change and writes the edit, the search goes unstable under a tight experiment budget and most trials are wasted on bad directions. The paper's diagnosis: LLMs are bad at allocating a scarce budget across many modification directions. They should propose hypotheses, not double as the explorer.
RecHarness splits the two jobs. First, a bandit router (Thompson sampling) picks the next direction to try from a predefined set of "edit arms," using historical validation feedback. Arms are actions like tuning the learning-rate schedule, changing dropout or weight decay, resizing embeddings, adding or removing layers and attention heads, swapping pooling, adding features, or changing the loss. They split into local arms (incremental tweaks within the current basin) and jump arms (structural changes for escaping to a new basin). Each arm keeps a Beta posterior: sample, take the largest, run the trial, update with a binary success signal. Second, given the chosen direction, the LLM agent produces a concrete hypothesis and the executable code edit. Third, to sustain long-horizon exploration there is a jump-basin mechanism: when the recent improvement rate over a window falls below a threshold (basin saturated), jump arms activate, and a jump is accepted only after R rounds of local retuning, so a structural change is judged after adaptation rather than on its raw first score. An "Experiment Skill" textual memory (incumbent, recent wins, dead ends) is fed to the LLM.
Offline: Amazon Reviews sequential rec (five backbones) and KuaiRec watch-time (three models). On average HR@10, the weak GRU4Rec baseline rises 85.85% (0.2685 to 0.4990) and the stronger HSTU 12.58% (0.4723 to 0.5317); all five improve and beat prior reported numbers. On KuaiRec's weaker TPM, WT-MAE drops 26.37% and WR-MAE 26.41%; the stronger GR still improves on all four metrics. The real claim is budget efficiency: 47.92% of RecHarness's non-baseline trials (23/48) beat the round-start best, averaging +5.06% and up to +24.00%, against TR w/ Random at 22.45%, TR w/ LLM at 21.74%, and w/o Bandit at 41.67%. In other words, it is the bandit routing, not letting the LLM pick directions, that makes trials pay off. Online: a 7-day A/B on 10% of traffic at a large short-video ad platform lifted ADVV 2.084%, revenue 0.534%, exposure 0.559%.
The honest takeaway is a division of labor: let a bandit do credit assignment over directions, let the LLM do code generation within a direction. That is more sample-efficient than "let the agent search freely," which is exactly where naive LLM-auto-ML capsizes. The online lift is small but it shipped, on huge traffic, which is the kind of evidence rec/ads teams accept. The method spans eight backbones and two scenarios, so it is not a single-template tuner.
The arms are human-predefined, so the search space is bounded; RecHarness automates selection within a space, not space design. The +85% gains on weak baselines look dramatic but partly reflect how bad the cold-start baseline is, not how good RecHarness is; the few-point gains on strong baselines are the more honest signal. The online lift is small (about 0.5% revenue) despite the traffic, and whether the bandit machinery is worth the complexity over a simpler human or random search is not fully isolated (w/o Bandit still reaches 41.67%). It is specific to recommender systems; there is no evidence it transfers to, say, LLM training.