NeurIPS benchmark: tree models still beat deep learning on 45 tabular datasets
tak3sh8 · x · 2026-10-27
The paper 'Why do tree-based models still outperform deep learning on tabular data?' (Grinsztajn et al.) benchmarks tree models against standard and novel deep learning methods across 45 tabular datasets.
Key findings: XGBoost and random forests remain state-of-the-art on medium-sized data (10K samples), even before counting their superior speed. The gap stems from inductive biases: trees are more robust to uninformative features, preserve feature orientation, and learn irregular functions more easily. The authors propose three challenges for tabular-specific neural nets and release a benchmark with 20,000 compute-hours of hyperparameter search data.
More from Research
- OpenAI claims its models refuted the Exponential Time Hypothesis; proposers are checking — ZeroCool86 · 2026-10-27
- AI-designed drug works in humans: aging looks 'like a list of fixable problems' — rand_longevity · 2026-10-27
- Pessimistic complexity theory hypothesis: faster algorithms must get uglier — Elijah_Meeks · 2026-10-27
- UMAP arcing explained: one giant coordinate dominates neural network activation maps — leland_mcinnes · 2026-10-27
- Kevin Bryan: maximizing raw accuracy is rarely the right way to train AI assistants — Afinetheorem · 2026-10-27
- Szegedy: a flaw in common training losses went unnoticed for over a decade — ChrSzegedy · 2026-10-27