A Self-Triggered Agentic Push Recommendation System
Zhao-Yu Zhang, Qingying Chen, Chunyuan Zheng, Jing Zhou, Jian Sun, Siqi Chen, Leiying Chen, Chuan Zhou, Huiyou Jiang, Xin Tao, Haoxuan Li, Zhouchen Lin
cs.IR
2026-08-03
ByteDance recasts Douyin push as a self-triggered loop: the system decides whether to send and when to wake itself, lifting active days 0.28% while cutting compute 79%.
Push notifications are one of the few recommendation surfaces where the system reaches users who never opened the app. The hard part is not what to send but whether and when. A billion-user app like Douyin cannot re-rank every user every second, so production systems pick one of two compromises: plan each user's push times offline (uplift modeling plus integer programming, then fire on schedule, blind to real-time state), or poll on a fixed short interval and decide in the moment, which burns compute when too frequent or misses the right moment when too coarse. Both are multi-stage pipelines that settle into local optima.
STEPS recasts push as a self-triggered closed loop. When the system wakes up it does two things at once: an execution agent decides whether to send now, and a planning agent decides how long to wait before waking itself again. Both are decision transformers (DT), which turn reinforcement learning into sequence prediction: given a target return, generate the matching action.
The planning agent has to emit a continuous time gap, and plain regression is unstable, so the authors recast it as ordinal regression, splitting 0 to 24 hours into 100 equal-frequency buckets and predicting which one. To stop the target-return signal from being drowned out by high-dimensional state features, they replace concatenation with a gated multiply (the return goes through an MLP, then multiplies the state element-wise). The tradeoff weight λ between positive and negative returns is sampled from a uniform distribution during training, so the live system can retune the balance without retraining.
The execution agent is a binary send-or-not classifier. Its key change is estimating Q-values with the Bellman equation instead of regressing the returns observed in offline logs, because the logged actions themselves are often suboptimal. A third lightweight filtering agent, distilled into a 3-layer MLP that deliberately carries no item features, prunes low-value requests before they trigger the expensive item-sorting pipeline (about 10× the cost), which is where most of the compute saving comes from.
14-day online A/B against Douyin's production offline-scheduling baseline:
| Metric | STEPS | Fixed-interval trigger |
| Active days | +0.2843% | −0.0670% |
| Push-disablement | −1.9089% | +0.0205% |
| Compute | −79.42% | +6.548% |
With compute matched, the fixed-interval approach does worse than the baseline. Ablations show the active-day gain is driven mainly by the planning agent, while nearly all of the 79% compute cut comes from the filtering agent (74.88% on its own). The spacing between pushes also gets healthier: bursts under 20 minutes drop 35.93%, and well-spaced 3-to-6-hour gaps rise 179.84%.
At billion-user scale, a 0.28% active-day lift is large in absolute terms, and because it lands alongside lower disablement and lower compute, this is a change that improves quality while spending less. The transferable idea is not any single number but the framing: treat "when to wake myself next" as a learned action, and a passive poller becomes an active planner. The "agentic" in the title should be taken with a grain of salt; it is really decision transformers, not LLM-based agents, which is precisely why it can run in real time at this scale.
A 0.28% relative gain only looks impressive against a billion users; the paper reports only relative percentages with no absolute baseline, so outsiders cannot judge the real payoff. Everything is evaluated on internal Douyin data with no reproducible public baseline. Safety still leans on hardcoded minute-scale thresholds (a locked window after a successful push or after a user goes active on their own), a sign the learned policy is not yet trusted with full freedom. The filtering agent is distilled from the execution agent at a small acknowledged cost in model quality.