ByteDance Deploys SequenceO1 at Full Traffic on Douyin, Compressing 100K-Long User Histories for Recommendation
_reachsumit · x · 2026-09-09
ByteDance introduces SequenceO1, an end-to-end framework for ultra-long (100K interaction) user behavior sequence modeling in recommendation, deployed at full traffic on Douyin.
The design follows a compress-then-reason approach:
- Sketch Attention (SA): uses learnable prototypes and prototype-wise normalization to compress raw history into a fixed-size, target-agnostic user representation, addressing storage, transfer, and training-throughput costs that scale with length.
- Target-conditioned Stacked Target-to-History Cross Attention (STCA): models complementary time scales with a recent 10K suffix for short-term signals plus the global sketch for long-term interests.
The paper argues that at 100K scale the challenge goes beyond attention complexity: prior approaches (truncation, multi-stage retrieval, compressed lifelong histories, short-train/long-infer extrapolation) either weaken end-to-end optimization or retain length-dependent cost. The cacheable sketch enables deployment under strict latency, memory, communication, and training-throughput constraints.
More from Research
- Timothy Duff's ECCV 2026 SfM-DL workshop slides on algebraic optimality for minimal solvers — ducha_aiki · 2026-09-09
- Drop a fixed batch proportion instead of per-sample tokens: capi author shares training trick — giffmana · 2026-09-09
- Adding Greek to a Cosmos3 VLA policy: bilingual training helps but lags far behind English — KIEFERSA · 2026-09-09
- Transformers encode a partner's expertise early but only act on it in later layers — Mika Okamoto · 2026-09-09
- Cadence uses a time-series foundation model for error-bounded lossy compression of demand data — Roberto Tacconelli · 2026-09-09
- Fourth Perception Test Challenge at ECCV 2026 pushes multimodal models on city-scale spatial intelligence — AjdDavison · 2026-09-09