Meta's CAP-Mix Replaces Hard Semantic IDs with Soft Prototype Mixtures for Generative Retrieval
_reachsumit · x · 2026-11-06
- Meta's paper (arXiv:2611.05799) tackles item representation in generative retrieval: hard Semantic IDs share parameters but discard embedding information, while continuous models avoid quantization but lack parameter tying.
- CAP-Mix, a flow recommender, represents each item as a soft mixture of shared prototypes plus an item-specific vector, trained jointly with assignments and prototypes, and retains the soft representation at inference.
- Pretrained info is optional: SID responsibilities serve as assignment priors, and full content embeddings can initialize item vectors without altering the initial flat table.
- In matched comparisons on three datasets, CAP-Mix beats a flat table on two sparse Amazon datasets with a small gain on Steam without pretraining.
More from Research
- ExPert paper inverts perturbation models to trace cell states back to causal causes — anshulkundaje · 2026-11-06
- Follow-up: gradient flow lets residual states encode information for far-future predictions — mlpowered · 2026-11-06
- Gautam Kamath on AI review: fun science experiment, but false positives can mislead people at scale — thegautamkamath · 2026-11-06
- UCLA's FlowPilot uses human preferences to cut sidewalk robot interventions by 40% — zhoubolei · 2026-11-06
- diffTLM: NeurIPS 2026 Spotlight differentiable vocal tract simulator reconstructs shape from voice — sainingxie · 2026-11-06
- AI model learns thousands of words from one child's audio — only with nightly replay, Science reports — jpillowtime · 2026-11-06