TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation
Wenqiao Zhu, Chao Xu, Haipang Wu, Ji Liu
cs.IR, cs.AI
2026-08-10
A learned policy selects the most informative tokens from each item's text for LLM-based sequential recommendation; 64 tokens match HLLM's 256, lifting NDCG up to 31.25% and cutting inference cost about 60%.
Using large language models for sequential recommendation (predicting the next item from a user's recent clicks) has an unavoidable cost: a single inference pass feeds the text description of every item in the user's history through the LLM. With history length S and n tokens per item, that is S times n tokens per step, expensive in both compute and memory.
A common economy is to keep only the first k tokens of each item's description. The problem is that most textual information sits later in the text: a book's genre or a game's mechanics are rarely in the opening tokens. One comparison shows how much this hurts: on Amazon Books, HLLM (a representative LLM-based recommendation framework) beats the traditional ID baseline SASRec by 24.4% using the first k tokens, but that gain collapses to 2.14% with random tokens. Which tokens you keep matters more than how many. More counterintuitively, taking the k tokens with the highest logits (a seemingly smarter rule) does no better than naive first-k, which the authors attribute to standard LLM attention being unidirectional and unable to capture collaborative signal in hidden states.
TSPORec asks whether a learned policy can pick the tokens that genuinely carry the most recommendation value from the full text.
A three-stage pipeline.
Stage one pretrains an LLM-based recommendation model in the usual way, with the InfoNCE loss (the contrastive objective that pulls positive pairs together and pushes negatives apart), split into an item LLM and a user LLM. The item embedding is the last-layer hidden state at a special [ITEM] position.
Stage two freezes the LLM backbone and attaches a small policy head to learn which tokens to keep. The hard part is defining "useful" without enumerating every subset (combinatorial explosion). The proxy reward works like this: for one item, the policy samples two different token subsets, plugs each into the user's history, runs them through the frozen LLM to get two user embeddings, and computes cross-entropy against the item's full embedding (with 128 negatives). Whichever subset gives lower cross-entropy locks onto the item more precisely for that user, so it is more informative. The reward is a +1 / -1 preference signal, and the objective maximizes expected reward.
One design point matters. The policy scores tokens with bidirectional query-key attention, aligning each token's hidden state with the item-level representation h{k+1} to get an importance score. This sidesteps the unidirectional-attention problem above and lets semantic and collaborative signal flow in together. Selection is done at chunk level (groups of c consecutive tokens, taking floor(k/c) chunks) for controllable granularity; smaller chunks do better, and the paper uses c=8.
Stage three uses the trained policy to pick high-probability chunks for every item, rebuilds the dataset, and retrains the LLM on the shortened inputs.
A theorem (Theorem 1) underwrites the recipe: lower cross-entropy means a user distribution closer (in KL divergence) to the true preference; chunks shared between the two subsets contribute no gradient to the policy; and the objective does raise the probability of informative chunks while suppressing the rest.
Two public datasets (Amazon Books, Pixel), two backbones (Qwen3-Embedding-0.6B, TinyLlama-1.1B), text truncated to 64 tokens. Metrics are Recall@K and NDCG@K.
Qwen3-Embedding-0.6B, 64 tokens, Amazon Books:
| Method | R@5 | R@10 | N@5 | N@10 |
| SASRec (ID baseline) | 3.38 | 5.09 | 2.24 | 2.79 |
| HLLM (first-k) | 4.16 | 6.29 | 2.80 | 3.48 |
| TSPORec | 4.37 | 6.55 | 2.94 | 3.64 |
On Amazon Books, gains over SASRec reach +29.29% Recall and +31.25% NDCG, averaging +29.43% across six metrics. On Pixel, the maxima are +19.63% Recall and +18.24% NDCG, averaging +16.79%. Swapping in TinyLlama-1.1B still beats HLLM by 4.12% on average, and feeding TSPORec-selected tokens to a different recommender (LLMinit) lifts it 3.76%, so the selected tokens carry transferable signal rather than backbone-specific bias.
On efficiency, TSPORec with 64 tokens matches or beats HLLM with 256. On an H100, 64-token inference takes 326 ms versus 843 ms for 256. With offline item embeddings, cost drops 63.4%; fully online, 61.3%. The price is training time, about 21.5 extra hours (8x H100) over a single-stage baseline, but the model trains once.
For recommendation engineers, the main obstacle to putting LLMs in production rec systems is inference cost, and this work cuts that cost at the input length without trading away quality. The more valuable observation is the counterintuitive one: which tokens matter more than how many, and naive first-k or top-k-logit both lose to a learned policy that carries collaborative signal. The recipe of a learned selector plus a proxy reward is not tied to recommendation in principle; any setting where feeding long text into an LLM is too expensive and only the information-dense fragments matter could borrow it.
Two parts are underexplained. First, the proxy reward rests on the assumption that lower subset cross-entropy means more informative, using the full embedding as the anchor, but the full embedding is itself trained on truncated text, a circular dependency the paper does not discuss. Second, evaluation covers only two datasets and two backbones, both text-rich recommendation settings; whether this transfers to industrial settings with sparse text and strong ID-based collaborative signal is untested. The extra 21.5 hours of training is waived off with "the model trains once," which does not hold for production systems that retrain frequently. Finally, the case study shows the policy favoring content words and filtering frequent function words, but this is post-hoc interpretation without an ablation proving that this preference is what drives the gains.