ScalarLens: Numerical Embeddings with Stable Coordinates and Contextual Responses for CTR Prediction
Heng Yao, Tianying Liu, Yulou Shu, Yong He, Chuan Yuan, Kaibin Qiu, Guowei Chen, Jiayu Zhao, Siyun Hou
cs.IR
2026-09-24
ScalarLens keeps a context-invariant numerical coordinate and a bounded contextual response; it ranks first in 25 of 27 CTR settings across 1,539 runs, with mean Criteo AUC 0.8113.
CTR models usually give a scalar one vector. That choice welds "where the value sits on the line" to "what it means in this sample." On the Criteo validation split, the same I7 interval (2,3] carries residual click evidence of −3.2, +3.5, and +5.8 percentage points across C17 groups after additive main effects are removed, and +6.8, +1.9, and −7.2 across I11 ranges. The number does not move. The sign does.
Production adds an operational tax. Numerical fields often come from different jobs on different cadences. A separately materialized normalizer has to stay version-locked between training and serving. ScalarLens treats learning from original units as a deployment constraint, not a stress test.
The module replaces only numerical embeddings. Categorical embeddings and the CTR backbone stay put. Each scalar takes two paths.
Stable coordinate: a monotone local mesh in the training range, DEER-style interpolation between adjacent nodes. Interval index and interpolation weight depend only on that field's scalar and its parameters. Context cannot move them. Equal values produce equal coordinates in every sample.
Contextual response: local coordinates and categorical embeddings are concatenated, RMS-normalized, and mapped by per-field heads to a drive δ and a gate g. A shared low-rank operator UV runs T=3 bounded updates with state clipped to [−1,1]. A readout concatenates the coordinate, the gate, and the transients, and emits only numerical vectors. Defaults are d=16, K=16, m=8, r=16.
The matrix is 3 datasets (AutoML-A/E, Criteo) × 9 backbones (DNN, DeepFM, WideDeep, NFM, PNN, DCN, DCNv2, AutoInt, FiBiNET) × 19 numerical representations × 3 seeds, 1,539 runs. The primary protocol keeps original scales. A second audit reapplies shared standardization.
On original scales ScalarLens ranks first in 25 of 27 settings and second in the other two, mean rank 1.074. Both second places are on AutoInt: 0.0005 behind DEER on AutoML-A, tied with B-spline at displayed precision on AutoML-E.
| Dataset | ScalarLens mean AUC | Strongest peer | Mean AUC gap |
| AutoML-A | 0.7139 | DEER 0.7069 | +0.0070 |
| AutoML-E | 0.8425 | PLR 0.8388 | +0.0037 |
| Criteo | 0.8113 | first on 9/9 backbones | +0.0012 |
Paired ΔAUC versus DEER / DAES / NaryDis is +0.0041 / +0.0055 / +0.0060 on raw scales, all significant after Holm-corrected Wilcoxon tests. The gap remains under shared z-scoring, so tolerance to raw scale is not the whole story. Dropping the response costs 0.0005–0.0091 AUC. Generic ContextFiLM does not reproduce the gain. In a controlled recovery study the focal coordinate moves by exactly zero while the response tracks categorical, numerical, and mixed context shifts. Optimized training latency is 0.74× DEER; inference is 1.13×.
Numerical embedding is framed as a measurement split: coordinates belong to values, predictive responses belong to value-in-context. For teams whose feature jobs are unsynchronized and who do not want extra normalized columns, this is a drop-in module. Absolute AUC gains sit in the 0.001–0.007 band, which CTR systems often care about, and the paper has no online A/B. Winning 25 of 27 settings is the transfer argument, not a claim of universal dominance.
No online test. Hyperparameters are frozen, and AutoInt is the weak spot. Some baselines are reimplementations, not official modules. +0.0012 AUC on Criteo is small; statistical significance is not business significance. Response dynamics still cost 13% inference versus DEER after optimization.