TWICE: Two-Clock, Two-Window Learning for Long-Horizon Conversion Prediction in Online Advertising
Kaiyuan Li, Kun Wang, Zhongbo Wang, Teng Sha, Ming Yan, Yanhua Cheng, Xialong Liu
cs.LG, cs.IR
2026-07-28
TWICE splits long-horizon ad CVR into a click clock and a conversion clock, routing late conversions to a separate delay model; recovers ~90% of the oracle gap, online revenue +2.5%.
Ad systems predict whether a click will convert (CVR), but conversions arrive late: an ad clicked today may not result in a purchase for days, or even 30 days. The model has to train on clicks that just happened, whose conversions have not come back yet, so the labels are incomplete. Waiting 30 days for labels to mature leaves the model stale.
Existing fixes (MISS, importance-sampling bias correction) make different trade-offs. Kuaishou tackles two problems at once: get recent clicks into training fast (a short observation window) while still predicting the long-horizon (30-day) CVR from clicks whose outcomes are not mature; and handle the fact that late-arriving conversions come from historical click cohorts of different ages, which biases the model when mixed. TWICE makes the process explicit by splitting it into two clocks.
The core is factoring long-horizon CVR into two separately trainable parts, po(x) = pv(x) F(o|x): the short-window CVR po equals the target-window CVR pv times the delay CDF F at o. The short-window label and the long-horizon target are bridged by the delay model.
The split exists because feeding late conversions straight into the CVR head injects information about the delay distribution into the will-it-convert judgment. Separated, delay information feeds only the delay model, and the CVR head sees clean short-window labels.
Offline, on two datasets, results are measured by RILL, the share of the gap between Vanilla (the myopic baseline) and Oracle (the hindsight, undeployable upper bound) that is recovered:
| Dataset | Clicks | CVR | TWICE AUC | vs strongest baseline | RILL |
| Criteo | 15.61M | 22.66% | 0.8413 | MISS 0.8395 | 89.7% |
| Industrial ads | 174.09M | 0.64% | 0.9185 | MISS 0.9160 | 90.4% |
TWICE recovers roughly nine-tenths of the gap to the undeployable Oracle, using a single 30-day checkpoint throughout.
Online A/B (Kuaishou's ad system, billions of requests a day): expected revenue +2.486%, revenue +1.858%, conversions +2.061%, all at p<0.01; calibration (PCOC) corrected from 0.961 to 0.990; prediction latency 36 ms and training time 4 minutes, both unchanged.
The telling evidence: TWICE's AUC gain over MISS grows with delay, from 0.00012 in the [0,1h] band to 0.00405 in (14d,30d]. The advantage really does come from handling delayed conversions better.
This is a textbook case of industry dissecting an overlooked bias structure. For anyone working delayed feedback in CVR/CTR, the two-clock decomposition with gradient isolation is directly portable: it needs no replay of historical clicks, its 4-minute training and 36 ms prediction are perfectly acceptable in production, and it lifted revenue online. Stripping delay information out of the CVR head is a design idea worth borrowing for any prediction task where labels arrive late.
The authors are explicit that the objective's exactness rests on two assumptions: pCVR mass is calibrated in aggregate within each delay context and click-time range, and converters within a context share a stationary delay distribution over the retained target window. In production these hold only approximately (G is deliberately compact, and each click-time score is frozen once logged).
The delay model can use only the compact G(x), not the full x, because full conditioning degenerates the arrival likelihood (exact-key aggregation creates singleton risk sets with zero gradient). That is an engineering compromise, so the delay distribution is modeled coarsely. The paper also stresses this is a plug-in conditional likelihood that corrects cohort exposure bias, not a claim of universally unbiased delay estimation.