UNIQUE: A Unified Retrieval and Ranking System for Large-Scale Feed Recommendation
Zhuang Liu, Yongkang Fu, Zuodong Yang, Guangxing Chen, Zonggang Wu, Yuqi Lu, Shouke Qin, Shantao Li, Maolin Wang
cs.IR
2026-09-20
UNIQUE jointly trains flat semantic codes and target-aware ranking in one Transformer. HR@10 beats UniGRF-HSTU by 12.52%; Mobile Baidu watch time +0.96%, distribution +1.08%.
Industrial feeds still retrieve then rank. Generative retrieval maps items to semantic codes and sidesteps a huge ID vocabulary, but hierarchical schemes such as RQ-VAE pass coarse errors down the stack. Head items hog the codebook; long-tail items share leftovers. Retrieval and ranking still train on different objectives and talk only through a candidate list, so ranking supervision never edits the retriever, and ranking cannot score items the retriever never sent.
UNIQUE ties feedback-aware single-layer flat quantization to an early-fusion backbone, so code generation and multi-task scoring share the user representation. Mobile Baidu's homepage, discovery page, and short-video retrieval have served it since January–February 2026. The venue is RecSys 2026.
Three modules. EQN (End-to-End Quantization Network) first runs a DSSM two-tower so user feedback (relevance, CTR, duration, completion) shapes item vectors, then nearest-neighbor quantizes them in one layer with EMA codebook updates. Assignment distance is multiplied by (usage / mean)^τ so hot codes cost more and cold codes cost less. Production uses 16,384 flat codes in place of a 128×128 hierarchical book of the same capacity.
UIGN encodes a user profile plus short, medium, and long-term behavior (public setup 1+100+200+200, length 501). TDN appends candidate item and code tokens to the same Transformer. Targets attend only to the user prefix, not to each other, and one forward pass emits both code logits and multi-task scores.
Training sums generation cross-entropy, discriminative BCE/MSE, the codebook two-tower loss, and quantization regularization. Inference predicts top codes, looks up items (N = 100 codes in production), inner-product coarse-ranks to about 10k, then fine-ranks grouped by code and sends M = 30 items per code downstream.
KuaiRand-Pure: 27,285 users, 7,583 items, mean length 43.47, leave-one-out against the full corpus. Codebook 1,024, flagged as a sanity check.
| Model | HR@10 | HR@50 | NDCG@10 | NDCG@50 | CTR-AUC |
| SASRec | 0.0304 | 0.0847 | 0.0189 | 0.0310 | N/A |
| TIGER | 0.0380 | 0.0902 | 0.0205 | 0.0378 | N/A |
| HSTU | 0.0453 | 0.1418 | 0.0273 | 0.0412 | 0.7150 |
| UniGRF-HSTU | 0.0511 | 0.1453 | 0.0298 | 0.0430 | 0.7168 |
| UNIQUE | 0.0575 | 0.1510 | 0.0310 | 0.0473 | 0.7252 |
Versus UniGRF-HSTU, HR@10 is +12.52%, HR@50 +3.92%, NDCG@50 +10.00%, CTR-AUC +0.0084 absolute. Dropping the generative loss hurts HR@50 by 9.01%; dropping early fusion by 7.42%; swapping flat codes for hierarchical by 5.89%. DIN/DCN are ranking-only at 0.6812/0.7037 AUC.
Production codebook: resource-count variance falls from 1510.85 to 389.57; mean top-1 category share inside a code rises from 65.43% to 78.08%. One-week full-traffic A/B (p<0.05):
| Metric | Relative gain |
| Total watch duration | +0.96% |
| Total distribution volume | +1.08% |
| User retention rate | +0.70% |
New users (registered ≤14 days) gain +1.44% watch time; high-activity (active >5 days/week) +1.11%; low-activity only +0.29%. UNIQUE supplies 22.5% of the retrieval pool. Tail items (bottom 20% by historical clicks, 1.82M items) reach a 1.04% exposure share. Peak about 10k QPS, mean latency 37 ms, P80 35 ms, P99 89 ms, inference MFU 44.23%, 400 L20 GPUs, about $0.0013 per 1k requests. Noon 15 s cache hits 37%; evening 30 s hits 60%. New items enter training within 5 minutes of first impression and are quantized within an hour.
Generative retrieval in production usually dies on codebook collapse and two-stage training. Flat codes with usage-weighted assignment are a more direct balancer than extra RQ layers. Early fusion lets ranking losses edit the user representation, so retrieval is no longer prior-only. Watch time and distribution rise together, which is not a smaller, safer slate. Both new users and heavy users move, so semantic codes help short histories and the same encoder still reads long ones.
Against MuSeR from the same lab, UNIQUE attacks cascade information loss rather than ultra-long multi-interest. They can coexist: one extends history, the other joins retrieve and rank. UNIQUE is 22.5% of the retrieval pool, not a replacement for the full ranker.
KuaiRand has 7,583 items and a 1,024-code book; it cannot carry the industrial codebook-balance story, which lives in the production analysis. UNIQUE is deployed as retrieval; fine ranking stays downstream, so some of the unified-training benefit may be truncated. Low-activity users barely move. N = 100, M = 30 is a cost compromise; N = 200 roughly doubles cost for little diversity. There is no online arm against MuSeR, and none that swaps only the codebook without joint training. Leave-one-out on a public set can leak popularity. The abstract says February 2026, the experiments section says January; the latter is the one that describes serving all online requests.