TraGR leads all 36 trajectory metrics by decoding each behavior's semantic IDs in parallel

From Tokens to Behaviors: Rethinking Generative Recommendation for User Trajectory Prediction

Jiaqi Wang, Hanchen Yang, Hengyu Zhang, Hongxu Ma, Wenqi Rao, Hongyang Wang, Zhongbo Sun, Renliang Qian, Weixin Chen, Wengen Li, Jihong Guan, Shuigeng Zhou, Qingpeng Cai, Peng Jiang, Hong Cheng

cs.IR

2026-11-12

TraGR decodes semantic IDs in parallel per interaction and passes a probability summary forward. It tops all 36 trajectory metrics on four datasets; MNDCG@10 rises 1.15% to 21.39%.

What problem this solves

Sequential recommenders usually score one next interaction. User behavior trajectory prediction (UBTP) asks for the next K interactions in time order: running shoes, then sports socks, then a water bottle.

Generative recommendation writes each item as a semantic ID (SID), a tuple of discrete codes, and decodes those codes token by token. On a trajectory, three mismatches show up. The codes inside one item and the jump to the next behavior share one autoregressive chain, so each interaction takes several dependent steps and the change between interactions has no state of its own. Later steps usually receive only the selected item. With socks at probability 50% and running shoes at 35%, passing the sock vector alone pulls the next step toward more socks. Teacher forcing trains on the observed prefix. Inference has to continue from the model's own predictions.

Method

TraGR, from Kuaishou, CUHK, Tongji, and Fudan, takes one full interaction as the generation unit. Item text is converted with optimized product quantization (OPQ) into a fixed SID: 16 tokens, 256 codes per codebook. The item vector is the mean of those token vectors, and the code assignment stays frozen. A 2-layer causal Transformer reads at most 50 history items and uses the last position as the initial behavior state.

The state advances once per predicted item. One residual head per SID position reads that same state, the heads predict in parallel, and the token probabilities are composed into a distribution over the full catalog. Sixteen tokens therefore do not need sixteen serial decodes. After an item is chosen, the feedback vector still mixes in a summary of alternatives. Summary weights use both the predicted probability and similarity to the chosen item, then a coefficient η blends the summary with the chosen item. Stage II and inference use η = 0.75. A shared transition network applies one residual update from the state, the feedback, and a step embedding.

Stage I supervises the next item and the K-step trajectory on observed prefixes, with η = 0. Stage II samples 8 of the model's own trajectories per history. The position reward is a hit on the observed item plus the training-set frequency of that transition, weighted by 0.25. Expected reward from later steps is credited to earlier sampled choices, centered by the average return of the other samples, and held near a frozen reference by a KL term with coefficient 0.5.

Results

Beauty and Cell Phones use a horizon of 3. ML-1M and Last-FM use 5. The split is chronological, and every method ranks over the full catalog. MHR and MNDCG are arithmetic means of the position scores. SHR and SNDCG are geometric means. SeqMatch@50 is the share of test trajectories in which every future item falls in the top 50 at its own position. TraGR is first in all 36 dataset-metric cells. Against the strongest baseline on each dataset, MNDCG@10 rises by about 1.15% to 21.39%, and SNDCG@10 by about 1.63% to 18.65%. Relative SeqMatch@50 gains are about 29.46%, 75.65%, 6.63%, and 12.02%, in dataset order. Main-metric gains have p < 0.05 over three seeds. On Beauty, the two reference numbers are MNDCG@10 and SeqMatch@50.

DatasetMHR@10MNDCG@10SeqMatch@50Reference
Beauty0.06290.03320.0435RPG-M: 0.0288 / 0.0336
Cell Phones0.04830.02440.0202EchoRec MNDCG@10 0.0201
ML-1M0.21440.11200.1545Best baseline MNDCG@10 0.0954
Last-FM0.33200.21110.2256MNDCG@10 margin about 1.15%

The 75.65% SeqMatch gain starts from 0.0115. Absolute coverage on Cell Phones is still 0.0202.

With no recurrent state, SNDCG@10 is 0.0219 on Beauty and 0.0521 on ML-1M. Ordered supervision that feeds the observed item moves those figures to 0.0258 and 0.0972. Zero feedback barely changes anything. Treating the future items as an unordered set and training with multi-label loss drops SNDCG@10 by about 25% and 23% from the history-only setup.

Full feedback pulls ahead of the chosen-item-only rule mainly on later steps. On ML-1M the relative HR@10 gain grows from about 5% at step 1 to 19.3% at step 5. On Beauty, step 3 shows a 31.0% relative HR@10 gain and an 18.2% relative NDCG@10 gain. Similarity reweighting adds another 5.34% SNDCG@10 and 7.65% SHR@10 on Beauty over probability weighting alone. Dropping the chosen-item anchor lowers the scores.

Continued supervised training, and cross-entropy on the model's own prefixes, both finish below the full Stage II objective. Removing either the collaborative-transition reward or the continuation credit lowers every reported score. Blocking gradients through the feedback path drops Beauty SNDCG@10 from 0.0296 to 0.0281.

Models trained separately at K = 3, 5, 7, and 10 keep TraGR first on SHR@10 and SNDCG@10 at every horizon on ML-1M. On Last-FM it leads SHR@10 throughout. The exception is SNDCG@10 at K = 3, where LPDO is slightly higher. All three methods degrade as K grows, and TraGR's relative drop is smaller. On one H200 at batch size 64, full-test inference is faster than LPDO with one denoising step and GCB with beam width 100, slower than greedy RPG, and uses less peak memory than GCB but more than LPDO and RPG.

Why it matters

SID tokens are decoded in parallel inside one interaction, the behavior state moves only between interactions, and the next step receives the distribution. The backbone is a 2-layer Transformer of width 448. Code is released.

A next-click ranker, and a basket model that ignores order, are different tasks. The multi-label ablation loses accuracy once order is removed. The gains depend on the dataset. Short sequences still have single-digit hit rates. On Last-FM, a catalog of 987 items, the MNDCG@10 margin is about 1.15%.

Limitations

Catalog scoring and feedback scale linearly with the number of items. Candidate pruning is left to future work. Beauty, the largest catalog here, has about 12,000 items. Cell Phones still covers only 2.02% of test trajectories at SeqMatch@50, so the 75% relative gain does not mean those trajectories are predicted well.

GCB has no public code. The table uses a reimplementation with RQ-VAE semantic IDs and beam width 100. Its MNDCG@10 is 0.0146 on Beauty, below SASRec at 0.0148 and far below RPG-M, and 0.0665 on ML-1M, below SASRec at 0.0727. The gap versus that reimplementation is weaker evidence than the gaps versus RPG-M, LPDO, and EchoRec. Beam methods are scored on decoded candidates. Invalid SIDs are dropped, and lists shorter than the cutoff are not padded. TraGR is scored on the full catalog. The paper states the protocol, and the comparison is still asymmetric.

Beauty keeps 5,123 of 22,363 users, and Cell Phones 3,216 of 27,879: only histories long enough to yield train, validation, and test trajectories. Last-FM sequences average length 1,176, while the encoder sees at most 50 items. Transitions absent from training receive zero collaborative reward. Training time and online results are not reported.

Terms

Source

What people are saying

Related papers

All paper explainers