NumEmbBench: first open benchmark for numerical embeddings in CTR finds no single best encoder
_reachsumit · x · 2026-11-04
Paper 2611.01090 introduces NumEmbBench, the first open benchmark for numerical feature embeddings in CTR prediction, released as a FuxiCTR extension.
- 18 configurations grouped into six mechanism families, evaluated on 3 datasets, 9 backbones, 5 seeds under a shared train-only z-score protocol
- DEER has the best mean rank, but six methods each lead at least one dataset-backbone setting — no single winner
- A matched native-scale test flips the winner in 11 of 27 settings; paired latency measurements show accuracy and inference cost need not move together
More from Research
- Noetra engineer's first project: building an image encoder from scratch for multimodal LLMs — HirokatuKataoka · 2026-11-04
- Japanese team trains image encoder from scratch to match SigLIP2, with better CJK performance — HirokatuKataoka · 2026-11-04
- SpecJudge author argues LLM-as-judge tools should never ask models for self-reported confidence — jokiruiz · 2026-11-04
- Tsinghua's BEMULA benchmark: no LLM agent memory system passes over 1.5% of forget requests — rohanpaul_ai · 2026-11-04
- KAIST AI's FactEmbed decouples query understanding to improve composed image retrieval — kaist-ai · 2026-11-04
- Intology's Locus auto-designs molecular models, beating GPT-6 and Claude agents — rohanpaul_ai · 2026-11-04