TAAL: Mitigating Early Beam Pruning in Generative Recommendation via Temporal Autoregressive Alignment
Lianjie Li, Zhiying Tu, Dianhui Chu, Hongliang Sun
cs.IR, cs.AI
2026-08-29
91.9%–96.6% of generative-rec failures happen in the first two SID steps. TAAL lifts Beauty NDCG@10 by 39.5% over LETTER via joint-prefix KL and PMI.
Generative recommenders encode items as hierarchical semantic IDs (SIDs) and decode the next item token by token. If beam search drops the ground-truth prefix early, the whole subtree is gone. Most follow-up work polishes the codebook or protects the single positive path. Harbin Institute of Technology breaks down LETTER-TIGER failures on Beauty, Instruments, and Yelp: at beam 20, 53.0%–71.1% of cases already miss at the first code, and the first two steps account for 91.9%–96.6% of retrieval failures, or 83.2%–92.4% of the full test set.
The bottleneck is the first two codes. Training is still pointwise next-token cross-entropy, which does not ask the model to cover the multimodal transitions seen in history. TAAL aims training and inference at those two steps.
SIDs still come from RQ-VAE, the backbone is still T5, beam search is unchanged. Only the first-two-step objective and the candidate scores change.
Training counts (c1, c2) joint-prefix transitions between adjacent items, excluding the last two validation/test items. The last Khist=5 events are mixed with exponential decay γ=0.7 into a history-conditioned soft target q12. Forward KL splits by the chain rule into an exact first-token marginal KL plus a conditional KL on Kmc=2 samples from q1, so every legal c1 need not be decoded. Forward KL is used because it penalizes paths that the data supports and the model puts near zero. Total loss is NTP plus α times this term, with α = 0.2 / 0.2 / 0.3 on Beauty / Instruments / Yelp.
At inference the model does not add log q12, which would boost globally frequent prefixes again. Pointwise mutual information divides the local transition by the global marginal, shrinks with τ=30, and adds β=0.02 times that lift to in-beam log-probabilities in O(beam) time.
Protocol matches DIGER: leave-one-out, full-catalog ranking. LETTER and APAO are rerun under the same protocol.
| Method | Beauty N@10 | Instruments N@10 | Yelp N@10 |
| LETTER (α=0) | 0.0347 | 0.0825 | 0.0228 |
| DIGER | 0.0372 | 0.0844 | 0.0227 |
| APAO | 0.0337 | 0.0811 | 0.0216 |
| TAAL, no rerank | 0.0477 | 0.0874 | 0.0286 |
| TAAL+PMI | 0.0484 | 0.0881 | 0.0293 |
Versus LETTER, NDCG@10 rises 39.5%, 6.7%, and 28.6%. Joint training alone is 37.5%, 5.9%, and 25.7%; PMI adds 1.5%, 0.8%, and 2.3%. Full-SID survival rises 16.6%, 3.9%, and 14.7%. Aligning only c1 lifts first-token survival and leaves c2 and full SID almost unchanged. A global-marginal control and a history-shuffled control beat NTP slightly at α=0.1 and fall below it at α=0.4; history-aligned TAAL stays ahead at both 0.2 and 0.4. As beam width shrinks from 50 to 5, relative survival gain grows from +9.3% to +39.4%. Main numbers use seed 42.
Failures in generative recommendation pile up at the widest SID branches, not only in a weak codebook. TAAL leaves the tokenizer and backbone alone, uses history-conditioned corpus transitions as coverage supervision, and strips global popularity with PMI. Relative gains grow as the beam narrows, which matches an early-truncation story rather than "search harder with a wider beam."
This is incremental, on three medium public sets, with one seed. Instruments only moves 6.7%; that catalog is already tight. PMI is a small rerank; the candidate set is built by the training term.
Transitions are first-order; multi-hop temporal structure is open. Three mid-size datasets do not show whether an industrial catalog has the same two-step cliff. The main table is one seed. β=0.02 is not the per-dataset maximum, only within 0.00017 NDCG of all three peaks. Even after TAAL, Beauty Hit@10 is 0.0856 from a 0.0671 NTP baseline; absolute recall under full-catalog ranking stays modest.