T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation
Yang Liu, Noel Loo, Ali Khanafer, Shuying Sun, Akshay Soni, Zhong Wu, Linjun Yang
cs.AI, cs.IR, cs.LG
2026-09-25
T-RoPE rotates by Unix time with multiscale frequencies and a non-stationary key; it leads all public metrics, lifts industrial NDCG 13–82% over Time RAB, CVR +0.33%.
Generative recommenders imported the Transformer and RoPE with it. RoPE rotates by token index. In recommendation, neighboring events may be seconds or months apart and still differ by index 1. Index RoPE is also time-translation invariant: the same history at Christmas and in June yields the same attention pattern, so seasonality never enters.
TiSASRec buckets intervals into attention logits. HSTU's Time RAB is the same idea, a scalar bias, not a change of geometry. TO-RoPE writes timestamps into RoPE angles, but query and key still share relative geometry, so the construction remains translation-invariant on paper.
T-RoPE, from Shopify with MIT and Liquid AI, keeps the RoPE interface and changes what the rotation means. Five pieces:
Theorem: timestamp RoPE with shared coefficients depends only on the gap. A non-stationary key with α^q+α^k ≠ 0 breaks translation invariance, and a yearly cycle can sit in one frequency plane. Extra FLOPs are linear in sequence length and head dimension.
Five public sets, one HSTU training stack, leave-one-out. HSTU+T-RoPE wins every metric on every dataset. On Amazon Books, HR@10 moves from Time RAB's 6.89 to 7.65 (+11.0%), about 8–12% across the five metrics versus the strongest baseline. ML-20M gains are about 1%; histories are dense and temporally regular. On Pixel8M, HR@10 moves from Time RAB's 9.14 to 11.47 (+25.5%); on Pixel1M, from 12.46 to 13.35 (+7.1%). The abstract's "78–130% on PixelRec versus the strongest baseline" is versus TO-RoPE, which is often weaker than Time RAB on PixelRec. That relative range should not be the headline. In the industrial ablation, dropping shifted query alignment costs about another 7% NDCG@50, on top of the frequency bank and the non-stationary key.
Industrial Shop data: 150M users, 4.8M items, more than 6B interactions, one-day temporal holdout. Versus the HSTU+Time RAB backbone:
| Metric | T-RoPE | Time RAB backbone |
| Recall@50 | 31.41 | 25.68 |
| NDCG@50 | 22.94 | 13.81 |
| MRR | 23.14 | 12.70 |
Dropping the multiscale bank sends NDCG@50 to 13.23. Dropping the non-stationary key costs about 4% NDCG@50. Removing Time RAB from the full model drops MRR from 23.14 to 16.89, so the relative bias and the rotary geometry add rather than replace.
Online A/B in the Shop app, control is the same model without T-RoPE: CVR +0.33% (p=0.037), order count +0.63% (p=0.094).
Generative recommenders should not borrow text RoPE as-is. The multiscale bank does most of the work; the non-stationary key is what puts calendar phase into the geometry. Time RAB still helps; it is not a fork in the road. The CVR lift is small and significant. Orders sit at p=0.094, directional only.
For teams already on an HSTU-class backbone, this is a drop-in positional encoding, not a new model.
Large percentages versus TO-RoPE on public data are easy to over-read; the strongest public baseline is Time RAB. Industrial data and code are private. The β range is a hyperparameter and may need retuning by domain. The order metric misses 0.05. Only HSTU is tested; whether TIGER-style semantic-ID generators gain the same way is unanswered. Unix seconds go straight into trigonometric functions out to 10⁸; stability notes live in the appendix.