Hybrid GPU-CPU Retrieval for Personalized Search at Ultra-Large Scale
Hao Fu, Jichao Sun, Baiting Zhu, Qiaoling Liu, Yan Shi, Cheng Lu, Liu Liu, Yubo Wang, Xin Yao, Xiangyu Niu, Xu Dong, Wenhan Lyu, Chiyao Shen, Yinjie Huang, Minglei Chen, Shuai Ding, Li Fan, Xiao Kong
KDD 2027 Applied Data Scien
cs.IR, cs.DC, cs.LG, cs.PF
2026-09-18
Meta co-serves a billion-doc GPU interaction path and a ~20× CPU inventory; vs the legacy CPU stack, DCG@20 rises 4.51% and GSRR 2.01%, with near-zero candidate overlap.
Personalized search over user-generated content pulls in two directions. Discovery queries need nonlinear interactions with behavioral history, so the same "food nearby" query prefers seafood for one user and fast food for another. Navigational and tail queries succeed only if a rare review remains retrievable. The source corpus is multi-trillion documents, then policy, quality, language, freshness, and dedup filters.
The paper calls this the personalization-scale paradox. Deep interaction belongs on GPUs, but HBM cannot economically hold the full online inventory. CPUs can hold inventory and cannot run the same interaction model on the latency path. SVFusion and FusionANNS split one ANN call across CPU and GPU. This system uses two model families and two independently selected inventories that meet only at aggregation. All authors are at Meta. The target venue is KDD 2027 Applied Data Science.
The contribution is heterogeneous co-serving, not a new model class. The GPU pathway retrieves over a curated pool on the order of a billion documents and pre-ranks with DeepFM-style interactions while candidates are still on the accelerator. The query tower concatenates a 384-d semantic embedding and a 128-d user profile into 512-d; the document tower emits a 128-d vector. Training jointly uses InfoNCE, Smooth L1 relevance, and BCE engagement. Serving follows SilverTorch: ANN, inference, and pre-rank stay on AMD MI300X, with sparse tables fetched from host memory over PCIe Gen5. One card held about 150-200 QPS at 30-40 ms P99 model-server latency, excluding network and downstream ranking.
The CPU pathway covers an inventory about twenty times larger (tens of billions) with a unified XLM-V two-tower at 384-d, interacting only by dot product. Embeddings live in a dedicated index, decoupled from latency-sensitive lexical posting lists. Serving switches nearest-neighbor scans to term-at-a-time eager evaluation and AVX-512 hit scanning. Centroids move from 64k per shard to a distributed 512k trainer, so index servers no longer cluster locally.
Either or both pathways can run per request and can be disabled independently. The aggregator deduplicates by document id, keeps source attribution, and hands a shared ranker the union. The GPU pool is selected for search value, not virality; an evergreen slice keeps under 0.1% of content older than one year.
A full-system account-level A/B against the legacy CPU-only stack, on the order of millions of accounts per arm per day, reports equal-day means over nine days:
| Configuration | DCG@20 | GSRR |
| GPU pathway (separate test) | +0.73% ± 0.15% | +1.04% ± 0.13% |
| CPU pathway (separate test) | +3.16% ± 0.15% | +1.20% ± 0.11% |
| Hybrid production | +4.51% [+3.99%, +5.04%] | +2.01% [+1.71%, +2.32%] |
DCG@20 is model-scored relevance, not human ratings. GSRR (Good Search Result Rate) marks a session as good if it contains a sufficiently long view or an explicit positive action. Every daily interval sits above zero. Pathway tests ran in different windows, so rows must not be compared across, and the hybrid lift is not the sum of the two rows.
Over three days where both dense pathways ran, Jaccard overlap is 0.75%, 0.44%, and 0.41% on head, torso, and tail queries. The shared-ranker input averages about 781 deduplicated candidates: 209 GPU-only, 47 dense-CPU-only, 500 lexical-only, and 26 overlapping groups. GPU retrieves 594 and passes 224 (37.6% reach); CPU retrieves 1,352 and passes 59 (4.4%). Production branch P50 is 196 ms lexical, 41 ms CPU, 151 ms GPU.
Dedicated indexing plus eager execution cut retrieval-stage compute 89.45% (9.48×). A 15-day experiment with 512k centroids and nprobe=256 cut CPU use 63.5% with no detected GSRR regression. A separate capacity plan for the same 256-d SQ8 broad-vector workload with three regional replicas puts accelerator capacity cost at about 4× the CPU plan.
Ultra-large personalized retrieval does not have to force one hardware tier to optimize incompatible objectives. Modeling depth sits on GPUs, inventory breadth on CPUs, behind a stable candidate contract with independently versioned inventories, models, deadlines, and rollback. For industrial search this is a depth-breadth architecture you can evolve one side at a time, with evidence from production A/B rather than offline recall curves. The matched capacity plan also says stuffing tens of billions of vectors into GPUs is a bad buy.
Pathway experiments change several components at once, so lifts cannot be assigned to hardware, ANN precision, model depth, or inventory size. The hybrid A/B validates the full operating point, not an interaction effect. Diversity at the ranker input is not Top-20 exposure or per-source engagement; that needs attributed impressions. Learned request-level routing is listed as future work and would need randomized pathway assignment. Subgroup quality and creator exposure are unproven. A predecessor semantic model that improved offline and hurt online viewing is the reminder that embedding, ANN config, online features, and downstream filters must ship as one bundle.