MuSeR: Scalable Long-sequence Recommendation with Multi-interest Modeling
Yongkang Fu, Beining Bao, Yu Jiang, Xiangyu Zhao, Hongyang Wei, Guangxing Chen, Zuodong Yang, Shantao Li, Zonggang Wu, Yuqi Lu, Shouke Qin, Hanmeng Liu, Maolin Wang
cs.IR
2026-09-20
MuSeR compresses ultra-long histories and splits multi-query interests with ERNIE/BGE text. Home-feed Recall@100 rises from 0.0714 to 0.0870; online DAU +0.26%, duration +0.89%.
Baidu APP's homepage, discovery feed, and short-video surfaces have to match user intent in milliseconds. A user can accumulate 10⁴–10⁵ actions across news, Q&A, and video: stable tastes mixed with conflicting intents. Production stacks truncate to a few hundred events and throw the long tail away. Sparse ID embeddings are thin for cold-start and short video. A Transformer on a 10⁵-length history is quadratic, and the usual SLA is under 300 ms.
MuSeR (Multi-interest Sequence Representation) does not sell a new primitive. It is a retrieval channel that makes ultra-long compression, multi-interest vectors, and multimodal alignment jointly servable, on top of the existing MGS system (HNSW-style multi-objective search).
Hierarchical temporal compression: keep the recent segment at full resolution, pool the mid-term every 16 neighbors, pool the early history every 64. Compressed length is Lc = Nr + (Nm − Nr)/16 + (T − Nm)/64. Production uses Nr = 800, Nm = 2000, Lc around 1,000. A 6-layer Transformer (d = 512, 8 heads) encodes the compressed sequence.
Multi-query interests: M learned queries attend over the encoding to produce M interest vectors, with orthogonality regularization λ = 0.01. Training asks every head to predict future actions; serving softmax-weights the heads against the candidate embedding. Six heads are the default; discovery gains more from extra heads than homepage (relative 0.0358 vs 0.0286).
Multimodal alignment: ERNIE-4.0-Turbo writes item summaries, distilled into ERNIE-Speed, then BGE (768-d) embeds them. ID and semantic vectors fuse from a 0.7/0.3 init. Short video can also use Visualized-BGE/VISTA.
Serving computes long-term interests asynchronously into a cache, runs only the short sequence online, and mixes them with a load-dependent β(t). Async refresh cuts about 60% redundant encoding. Retrieval is hierarchical beam search on an HNSW graph: Euclidean expansion at the bottom, a DNN scorer for relevance, engagement, and conversion at the top. Graphs build on CPU; traversal and scoring run on GPU or Kunlun NPUs. Versus flat HNSW, end-to-end retrieval latency drops 27% and engineering cost more than 35%.
Public data are Amazon 2023 Instruments / Scientific / Video Games, 5-core, sequences truncated to 50, which is not the ultra-long setting. MuSeR beats the strongest baseline NANN on all three; Instruments Recall@5 goes from 0.0373 to 0.0381 (p<0.05), Games Recall@10 from 0.0910 to 0.0919. The public margin is tiny.
Industrial offline (100M users, 10M items, one month of logs):
| Method | Home R@100 | Home R@500 | Discovery R@100 | Discovery R@500 |
| SASRec | 0.0389 | 0.1176 | 0.0569 | 0.1372 |
| NANN | 0.0714 | 0.1982 | 0.1012 | 0.2287 |
| TIGER | 0.0511 | 0.1654 | 0.0721 | 0.1881 |
| KuaiFormer | 0.0524 | 0.1701 | 0.0818 | 0.1932 |
| MuSeR | 0.0870 | 0.2232 | 0.1377 | 0.2534 |
Ablations on Home Recall@500: uncompressed long sequence 0.2351, short truncation 0.1749, hierarchical compression 0.2232. Multimodal alignment lifts Home from 0.1636 to 0.2232 (+36.4%) and Discovery from 0.1892 to 0.2534 (+33.9%). Beam search (0.2232/0.2534) sits close to brute force (0.2328/0.2671); HNSW is faster with a larger recall drop.
Online A/B rolled from homepage in early 2025 to discovery and short video, user-hash buckets, downstream rankers frozen. Homepage feed DAU +0.26% and total session duration +0.89% (both p<0.05). Full deploy in August 2025. The authors treat a gain of this size as meaningful on a mature multi-source retriever.
Industrial retrieval is usually blocked by serving, not by another attention variant. Hierarchical compression trades 0.0119 Recall for deployability; multimodal alignment moves industrial recall far more than the 0.001 public Recall@5. Async caches and hierarchical beam search are the reusable pieces. For teams already on HNSW multi-objective retrieval, this is an integration checklist, not a new loss.
Public sequences are truncated to 50, so they do not test the ultra-long claim, and the Amazon lead is in the fourth decimal. The authors say they are not proposing a new primitive; the academic delta is modest. Online reporting is DAU and duration only, with no per-scenario CTR/CVR table and no "MuSeR as the sole retriever" arm. MGS internals are deferred to another writeup. Head count and Lc optima come from one industrial sweep. The ERNIE summary stack is Baidu-specific; a port needs a different LLM and a cost redo.