One Table, Many Queries: Benchmarking Generalist Tabular Models Across Generation, Imputation, and Prediction
Yuanrui Wang, Kaifang Li, Xuanyue Li, Ziyun Li, Yan Lu, Xingxuan Zhang, Peng Cui
cs.LG
2026-11-09
OTBench runs 46 tabular methods on 56 datasets across generation, imputation, and prediction: multi-task gains don't add up, and frozen foundation models already hold their own.
One table, three audiences: generative modelers synthesize whole rows, imputation folks fill missing cells, prediction people model a target. Each task has its own benchmarks and modeling traditions, yet all three are queries against the same joint distribution: generation samples the joint, imputation and prediction sample different conditionals. Whether those capabilities can be shared, and whether training on one objective helps another, had never been measured under aligned data and evaluation contracts. The question turned practical once tabular foundation models (pretrained on many tables, no per-dataset fine-tuning) such as TabPFN, TabICL, and LimiX started exposing several of these interfaces at once. StableAI and Tsinghua built OTBench to answer it: 56 datasets, 46 registered methods, 20 capability axes.
OTBench casts every task as the same query: given observed variables O, queried variables Q, and protocol C, a model approximates P(ZQ | ZO). Generation is the empty-O case, imputation queries feature conditionals, prediction queries the target conditional. All tasks share one 80/20 train-test split per dataset, with explicit contracts on what each task may see.
Dataset construction is guarded against selection bias: from 492 OpenML datasets cited across 24 studies, hard size filters leave 377; 113 method-independent meta-features (cleaned to 72, compressed by PCA to 41 components) feed a k-means clustering into 50 groups, the nearest nonduplicate candidate to each centroid is kept, 31 same-source versions are suppressed, and six structural-causal-model datasets join as a separate track. No evaluated method's results touch the selection.
Metrics go through a redundancy audit and collapse into 20 semantic capability axes (fidelity, C2ST, alpha-precision, TSTR gap, privacy diagnostics, and so on) that deliberately stay separate rather than merging into one score. Three analyses stack on top:
Rank alignment: imputation and prediction agree through numerical recovery, dependence preservation, and downstream utility. Generation's fidelity and C2ST axes correlate negatively with all four classification axes in a direction-preserving way, while TSTR gap and global utility correlate positively with every prediction axis. Looking real and being useful are different competencies.
SAE: adding the recipient's own objective is the strongest single intervention in the study, positive on all 19 displayed axes (0.122-0.445 for generation, 0.159-0.317 for imputation, 0.098-0.456 for prediction). Among auxiliary objectives, generation has the broadest coverage: positive on all six imputation axes and five of seven prediction axes. Adding prediction lifts synthetic-data utility (P→G TSTR gap +0.092, positive for four of five backbones, though the pooled interval [-0.041, 0.225] still crosses zero).
CAE: benefits do not add. With prediction present, adding G and I is sub-additive on all 12 recipient axes (six generation, six imputation), seven of them passing the full robustness criterion.
Fixed identities split by capability:
| Task | Axis | Leader |
| Generation | fidelity/C2ST/alpha-precision | Gaussian Copula |
| Generation | TSTR gap/global utility | TabPFN-3 |
| Imputation | numerical point accuracy | TabPFN-3 |
| Imputation | numerical distribution/association/utility | TabICLv2 |
| Imputation | both categorical axes | TabSyn |
| Prediction | every displayed axis | LimiX-2 |
Generalist versus specialist portfolios splits by profile. On downstream utility, TabICLv2 beats its portfolio by 0.157 worst-task percentile (95% CI [0.078, 0.229]) and TabPFN-3 by 0.142 ([0.051, 0.227]). On fidelity, portfolios win across the board: the categorical-fidelity portfolio leads every generalist by 0.261-0.396. The objective-arm Pareto frontier contains {G, GI, GP, IP, GIP}; the all-objectives GIP arm holds 94% bootstrap frontier membership, a competitive compromise rather than a uniform winner.
For practitioners the selection rule is now concrete: if synthetic data feeds downstream predictors, generalist foundation models already compete or win; if distributional fidelity is the requirement, specialists still win. For anyone training unified models, the ordering is clear: add the recipient task's own objective first, since it beats any auxiliary objective; generation is the most broadly useful auxiliary objective; and objective benefits should not be expected to stack linearly, since combinations behave differently per backbone.
For the foundation-model agenda, the remaining gaps are precisely located: categorical reconstruction and faithful joint sampling. Prediction and numerical point imputation are already handled well by frozen models.
The authors' own list: the causal-transfer study covers only five datasets (one regression) and five backbones, and auxiliary effects are exploratory and backbone-dependent; the categorical-fidelity profile cohort has three datasets; the rank study uses target-observed imputation while the transfer study hides the target, so the two contracts do not transfer directly; frozen-model results reflect the inference system and do not isolate the contribution of pretraining; SAE and CAE measure rank changes, not mechanisms.
One more piece of context for reading the leaderboard: LimiX-2, which leads every prediction axis, comes from StableAI, the same team behind the benchmark, with overlapping corresponding authors. The numbers are public and recomputable, but the context belongs with them. The wide interval on the P→G effect also means the claim that prediction helps generation is not settled yet.