Compote: Annealed Multi-Hypothesis Training Prevents Head Collapse in Sequential Recommendation
_reachsumit · x · 2026-10-29
A new arXiv paper introduces COMPOTE (Collapse-Resistant Multiple hyPOThesis lEarning) for sequential recommendation, addressing the fact that next-item prediction is inherently ambiguous while standard recommenders compress alternatives into a single prediction.
- Method: a multi-head transformer trained with an annealed objective that shifts from uniform supervision to winner-takes-all assignment, forcing each head to specialize in a distinct catalogue region and avoiding head collapse seen in conventional multi-head architectures.
- Inference: an efficient, normalizer-free max-similarity rule.
- Theory: in the population case, optimal solutions partition the conditional next-item distribution under a max-entropy quantization principle, pooled scores preserve Bayes-optimal ranking, and a generalization bound scales logarithmically with the number of hypotheses.
- Experiments: on three Amazon Reviews 2023 benchmarks, COMPOTE beats strong ID-based, generative and multi-interest baselines using item IDs alone; matched-capacity ablations confirm gains come from the annealed objective rather than capacity, ensembling or routing.
More from Research
- AI Audit Finds 2,035 Errors in 138 Papers by Economists Saez and Zucman — sebkrier · 2026-10-29
- Granary-v2 open speech model expands to 48 languages with open data and pipeline — rdesh26 · 2026-10-29
- Tinker launches Dimuon, a Muon-based optimizer that beats AdamW on LoRA fine-tuning — xennygrimmato_ · 2026-10-29
- Six CUDA kernel optimizations squeeze another 1.78× speedup out of 3DGS training — gkopanas · 2026-10-29
- Cross-lingual LLM effects paper wins Outstanding Paper award at EMNLP 2026 — anas_ant · 2026-10-29
- Lie detector compiled into Qwen3.5 weights flags 91% of lies with zero false positives — teortaxesTex · 2026-10-29