Densing law for user representation learning: tokenizing behavior breaks billion-scale scaling bottleneck
_reachsumit · x · 2026-08-25
The paper proposes a User Behavioral Densing Law characterizing the quantitative relationship between data scale and minimum sufficient tokenization capacity. On billion-scale Alipay data, raw data scaling hits a bottleneck, while tokenization yields sustained gains. They find an approximately linear log-log relationship between minimum sufficient tokenization capacity and input data size, with slope varying by method and data source.
More from Research
- Statistical Rethinking 2026: 20 Lectures on Causal Inference & Scientific Modeling — RichmanRonald · 2026-08-25
- Simulation-Based Inference: Neural Nets Solve Probability Black Boxes — burkov · 2026-08-25
- WIEN-INR: Neural Representation for Lossless Scientific Data Compression — bravo_abad · 2026-08-25
- Alibaba Proposes ERPO: Environmental Regularization for Stable LLM RLHF — alibabagroup · 2026-08-25
- NeurIPS 2026 workshop calls for papers on AI failure modes in biology — anshulkundaje · 2026-08-25
- SimCLR & MoCo vs. incremental science — 3scorciav · 2026-08-25