Embedding Items at Scale: Comparing GNN-Based and ID-Based Item Embeddings in the Yandex Ecosystem
Sergei Makeev, Artem Matveev, Vladimir Baikalov, Kirill Khrylchenko
cs.IR
2026-07-29
On Yandex Market and Music at full scale, end-to-end ID embeddings suffice and GNN pretraining isn't worth the extra cost; only on the low-data Lavka setting does pretraining help.
Transformer-based sequential recommendation models take a user's interaction history and predict the next item, and how items are embedded decides the quality. The field has long split into two camps. One uses pretrained item embeddings, typically learned ahead of time from a user-item interaction graph with a graph neural network (GNN) and then frozen as features. The other goes end-to-end, hashing item IDs into embeddings trained jointly with the transformer. The problem is that no one had compared these two paths on quality and cost together, at real industrial scale. This paper is Yandex's case study to do exactly that.
They compare across three of their own systems: Yandex Market (e-commerce) and Yandex Music (streaming), both production-scale, plus a low-resource Yandex Lavka (grocery) setting with public data and code.
The model is the production two-tower design: a user tower that's a transformer over the interaction sequence, and an item tower that's a residual network combining the BPE-tokenized title with the item embedding (GNN or ID). On the GNN side they test transductive TwHIN (learns a vector per node, needs all nodes at training time) and inductive MultiBiSage (learns a neighbor-aggregating function, handles new items). On the ID side, multihash maps item IDs into an O(10^6)-entry, 64-dim table. At evaluation, the transformer's score feeds the production CatBoost ranker as a feature, and they report the relative metric lift of adding that feature, with Wilcoxon significance at p<0.01.
At scale the story is consistent. Relative nDCG lift on Yandex Market across three surfaces:
| Item embedding | Discovery | Cart | Retargeting |
| None (content only) | +0.506% | +0.103% | +0.565% |
| TwHIN (GNN) | +0.790% | +0.151% | +0.943% |
| MultiBiSage (GNN) | +0.565% | +0.122% | +0.651% |
| ID embeddings (end-to-end) | +1.238% | +0.215% | +1.486% |
| TwHIN + ID | +1.273% | +0.235% | +1.522% |
End-to-end ID embeddings beat both GNNs. Combining TwHIN with ID edges slightly higher on every column (Discovery 1.238 to 1.273), but the authors explicitly say that incremental gain isn't worth running a separate GNN pretraining stage. Yandex Music tells the same story: ID embeddings (pair accuracy +0.699%) clearly beat TwHIN, and fine-tuning TwHIN doesn't help.
Low-resource Lavka flips: TwHIN (fine-tuned or not) edges out end-to-end ID on nDCG@5 (0.337 / 0.342 vs 0.333). On compute, ID embeddings take 40-70 hours, TwHIN takes 5, MultiBiSage takes 52, all on 8 A100s.
For recommendation practitioners, this is a directly cost-saving conclusion: if your interaction data is already at industrial scale, skip the separate GNN pretraining stage for item embeddings and just train ID embeddings end-to-end with the transformer; it's better in quality and shorter in pipeline. GNN pretraining only earns its keep when data is scarce.
The authors don't run a dedicated limitations section, but several boundaries stand out. GNN embeddings are computed only for the most popular slice of items (17% at Market, covering most interactions); long-tail items collapse to a shared embedding, while the ID scheme has no such cutoff, which structurally favors ID. The Lavka low-resource setting lacks visual and textual features, limiting its representativeness. Market and Music evaluate the transformer score as a CatBoost feature, not an end-to-end ranking metric, so the conclusions are strictly about feature contribution, not absolute online metric highs. GNN hyperparameters weren't fully tuned under compute limits, so whether the not-worth-it verdict softens under a more expensive GNN config goes unanswered.