HELIX: Purified and Unified - Rethinking Feature Interaction and Sequence Modeling for Large-Scale Recommendation
Yuntao Zheng, Miao Zhang, Yadong Ding, Yanchuan Tang, Lixiyu Chen, Hao Wang, Quan Li, Shiying Cai, Yue Lin, Jiayu Li, Yu Feng, Wentao Yang, Rongkun Xing, Jiekai Wang, Mingge Zhang, Feiling Gong, Xiang Gao, Jinyu Dong, Yajing Zhang, Pengfei Ren, Yinzhou Wang
cs.IR
2026-09-29
HELIX interleaves sequence retrieval and feature mixing with a reusable user K/V cache. On TikTok e-commerce it lifts CTR AUC +0.21% vs RankMixer and ~6% GMV per user.
Industrial ranking has to grow along two axes at once: high-order interaction over user, item, context, and cross features, and sequence modeling over long, multi-type behavior histories. FM, DeepFM, and DCN work at moderate scale, then saturate as width and depth grow. DIN, DIEN, and DSIN made candidate-conditioned aggregation standard, but the aggregator is shallow, so extra sequence capacity does not land.
Each axis later got its own scalable backbone. Wukong, RankMixer, and TokenMixer-Large tokenize heterogeneous embeddings and run residual blocks. HSTU, Ultra HSTU, and STCA put heavier attention on the sequence side. Grow either axis alone and the other becomes the ceiling. InterFormer, Kunlun, and OneTrans share a backbone, and they hit three production constraints. Self-attention is a weak fit for heterogeneous feature tokens. Folding retrieval and interaction into one step spreads mix-token attention across non-sequential inputs. Bidirectional updates make user-side states depend on the candidate, which blocks M-FALCON at serving time and user-level training at train time.
A deeper user encoder that reruns for every candidate will blow the compute budget first.
HELIX is described as purified and unified. Unified: sequence retrieval and feature interaction alternate along depth, not a two-stage pipe. Purified: LHUC, DCN-M, FM modules, cross-layer residuals, and auxiliary losses are dropped, leaving retrieval and interaction.
Three streams. MixTokenizer turns non-sequential features plus mean-pooled sequence summaries into M mix-tokens, each with its own LayerNorm and FFN. That LayerNorm is the only one in the model; the rest is RMSNorm. SeqTokenizer projects U-only histories (candidate-independent) and U×C histories (candidate-conditioned similarity and item-to-item sequences) separately, adds positions inside each source sequence, then concatenates, so no fake global order is imposed across sequence types. Learnable [SINK] tokens mark the start of each stream.
One-way flow:
Information never flows back into the user cache. Every candidate in a request shares it; every training sample of the same user shares one encoder pass. Attention uses QK-Norm, grouped-query attention, and gated attention.
Internal TikTok e-commerce video data, relative to the production RankMixer + Transformer baseline (210M parameters, 5.75G FLOPs per sample). The paper treats a 0.03% relative AUC lift as practically meaningful.
| Method | CTR ΔAUC | CTR ΔUAUC | CVR ΔAUC | CVR ΔUAUC | Params | FLOPs |
| RankMixer + Transformer | 0 | 0 | 0 | 0 | 210M | 5.75G |
| RankMixer Large | +0.04% | +0.15% | +0.03% | +0.19% | 1069M | 27.50G |
| OneTrans | +0.15% | +0.61% | +0.10% | +0.66% | 361M | 39.27G |
| HELIX-L | +0.21% | +0.82% | +0.17% | +1.38% | 358M | 35.28G |
RankMixer Large grows non-embedding parameters to about 1.07B and only adds 0.04% CTR AUC. HELIX-L sits in the same parameter band as OneTrans, wins on all four quality metrics, and uses slightly fewer training FLOPs (35.28G vs 39.27G). The OneTrans baseline is a post-publication production build.
S through XXL scale width, depth, and the retained sequence-token budget together. Versus HELIX-S:
| Size | CTR ΔAUC | CVR ΔAUC | Params | FLOPs |
| S | 0 | 0 | 46M | 2.31G |
| M | +0.24% | +0.07% | 99M | 7.90G |
| L | +0.34% | +0.18% | 358M | 35.28G |
| XL | +0.39% | +0.20% | 740M | 72.41G |
| XXL | +0.45% | +0.27% | 1050M | 130.24G |
A power law fits log(ΔAUC) against log FLOPs: slope 0.085 and R²=0.925 on CTR, slope 0.132 and R²=0.994 on CVR. XXL is still not saturated. RankMixer is competitive at low FLOPs and then lags. OneTrans sits below HELIX-L despite a larger compute budget.
Replacing decoder self-attention with MPTF at matched size adds 0.05% CTR and 0.02% CVR AUC. Restricting mix-tokens to attend only the cache, not themselves, adds 0.02% on both. A last-layer cache versus per-layer caches adds 0.01% CTR and leaves CVR flat, and it decouples sequence depth from interaction depth. Non-parametric Gate-Up-Norm adds 0.02%/0.01% and keeps SwiGLU products inside the FP16 range without SoftCap. Gating after the value projection (0.03%/0.02%) beats gating after SDPA (0.02%/0.01%). QK-Norm adds 0.03%/0.01%. GQA matches multi-head AUC and shrinks the cache. Encoder LayerScale adds 0.01% CTR.
HELIX-L versus RankMixer + Transformer in a full-traffic A/B test: +2.0591% e-commerce video clicks per user, +5.9932% GMV per user, +6.0964% paid orders per user, +5.4624% GPM.
If the encoder is 95% of per-instance compute and a request has 100 candidates, Amdahl's formula gives about 16.8×. That is a formula, not a measured serving latency.
The setting is large-scale e-commerce video ranking: many candidates, long behavior histories, user-side compute that must be amortized. Teams already on RankMixer, OneTrans, or HSTU-style stacks get a concrete split: attention retrieves from sequences, MPTF mixes heterogeneous features, and the user history is a one-way cache that can be deepened because M-FALCON and user-level training pay for it.
The conclusion says the same cache could be shared by generative retrieval, pre-ranking, and ranking. No cross-stage numbers appear in the paper.
Without this traffic and this serving stack, a 0.2%-scale relative AUC is hard to transfer. This is incremental industrial scaling, not a new task.
There is no Limitations section. Every number is from internal TikTok e-commerce video logs. Absolute AUC is withheld. No public data or code. Related work cites InterFormer, Kunlun, HSTU, Ultra HSTU, and STCA; the tables only compare RankMixer and OneTrans, and OneTrans is an optimized production fork that outsiders cannot check.
Most ablations sit at 0.01%0.05%. Under the paper's own 0.03% bar, half of them are on the edge or below it. The scaling fit uses five HELIX points; RankMixer has two, so this is not a cross-architecture proof that joint scaling always has a better slope. The 16.8× figure assumes p=0.95. MixFormer still depends on per-candidate U×C sequences, so the amortizable share may be lower.
"Purified" drops auxiliary modules, then the backbone still stacks gated attention, QK-Norm, LayerScale, Gate-Up-Norm, and GQA. Those pieces stabilize training; they are not "only two operations." Offline XXL still improves; production used L. Latency and stability at larger sizes are unreported.