TabPFN-3.5 Beats TabFM by 84 Elo in a Tenth of the Time, With Yann LeCun Among the Authors

TabPFN-3.5: Technical Report

Benjamin Jäger, Nick Erickson, Léo Grinsztajn, Felix Birkel, Klemens Flöge, Oscar Key, Kürşat Kaya, Jonas Kübler, Adèle Frankel, Tobias Schröder, Anurag Garg, Jan Hendrik Metzen, David Salinas, Simon Bing, Kristina Collins, Tuana Çelik, Vahid Balazadeh, Lydia Sidhoum, Tomás Pereda, Brendan Roof, Andrej Tschalzev, Siyuan Guo, Philipp Singer, Lennart Purucker, Jake Robertson, Marie Salmon, Philipp Jund, Jerry Chen, Diana Kriuchkova, Arthur Cahu, Eliott Kalfon, Adrian Hayler, Georg Grab, Vitor Monteiro, Lilly Wehrhahn, Dominik Safaric, Clara Cornu, Alan Arazi, Rylee Grace, Simone Alessi, Mihir Manium, Bernhard Schölkopf, Yann LeCun, Madelon Hulsebos, Sauraj Gambhir, Noah Hollmann, Frank Hutter

cs.LG, stat.ML

2026-09-16

TabPFN-3.5 tops all 7 benchmarks it's tested on, beating TabFM by 84 Elo points at one-tenth the compute time; weights are open but restricted to non-commercial use.

What problem this solves

Tabular foundation models already dominate clean, i.i.d. benchmarks against gradient-boosted trees, but real business data rarely looks that clean: training and test sets can come from different countries or hospitals (grouped shift), test data can arrive after the training window (temporal shift), and tables often carry high-cardinality categorical columns, heavy text fields, or hundreds of features. Prior tabular foundation models still lost to tuned conventional models in these messier regimes. TabPFN-3.5 tries to carry the tabular-foundation-model lead from clean benchmarks into the conditions practitioners actually face.

Method

The architecture largely keeps TabPFN-3's design but revises three things:

• Upgraded per-cell encoding. Beyond the standardized numeric value, each cell gets two extra signals: a learned-frequency Fourier encoding, which captures fine numeric differences and helps especially with high-cardinality categorical columns, and an in-context Empirical Cumulative Distribution Function (ECDF) feature, where each value is ranked against the training rows of its own column and mapped into [0,1], so a skewed feature and its logarithm produce identical ECDF terms, and heavy tails that standard clipping would crush get spread out evenly.
• A single checkpoint trained jointly on classification and regression. Only the label encoder and output head are task-specific; the encoder, aggregator, and transformer backbone are all shared, cutting training cost.
• Doubled model width (512 to 1024), paired with grouped-query attention so the KV cache size stays roughly the same at inference, single-row prediction speed matches TabPFN-3, with only large training sets seeing roughly 2x slower forward passes.

The synthetic prior was also retuned to generate more high-cardinality, wide, and grouped-data scenarios.

Results

Across 7 benchmarks, TabPFN-3.5 (or its variants TabPFN-3.5-Thinking / TabPFN-Rel) ranks first on every one:

On TabArena, TabPFN-3.5 beats TabFM by 84 Elo points using one-tenth of its total compute time, and beats AutoGluon 1.6 extreme by 130 Elo points in a fifth of the time. The faster variant, TabPFN-3.5-Fast, runs 3x faster than TabPFN-3 while staying about 150 Elo points ahead; the paid TabPFN-3.5-Thinking mode is 12x faster than the previous generation's Thinking mode.

Why it matters

For teams working with messy tables, text fields, high-cardinality categories, grouped or temporal shift, TabPFN-3.5 is the first tabular foundation model that carries its advantage into those conditions rather than falling back to hand-built features. The licensing matters here: the open-weight release is restricted to evaluation and non-commercial use, production deployment requires a separate enterprise license, and the stronger Plus and Thinking variants are API-only with implementation details the report explicitly declines to disclose.

Limitations

The report has no dedicated limitations section, but the results themselves mark the boundary: on grouped and temporal shift, and on the largest datasets, tuned and ensembled MLPs still lead, TabPFN-3.5 narrows the gap substantially rather than closing it. The Plus and Thinking variants' inference optimizations are proprietary, and the report states their implementation details are intentionally not described, which means outside parties can't reproduce or audit their real performance.

Terms

• BeyondArena: a tabular benchmark built specifically to test grouped shift (e.g. across countries or hospitals) and temporal shift (test data occurring after training), as a counterpoint to standard i.i.d. benchmarks.
• ECDF (empirical cumulative distribution function): converts a value's relative rank within its column into a number in [0,1], making it more robust to outliers and skew.
• KV cache: cached key/value vectors for already-processed training rows during Transformer inference, avoiding recomputation at test time, a key reason tabular foundation models can predict in well under a second.

Source

• arXiv 2609.17895v1
• PDF
• Hugging Face

What people are saying

• Frank Hutter on TabPFN: How Tabular Foundation Models Predict in a Single Forward Pass Machine Learning Street Talk
• Frank Hutter on why deep learning failed on tabular data for a decade Machine Learning Street Talk

Related papers

• Qwen3-8B hits 41.5% exact-match tool selection, beating 402B Maverick
• DeepMind fixes two numerical bugs in softmax and cross-entropy, eliminating Slingshot loss spikes
• Unsupervised partner D rises from 61.3% to 85.5% when JEPA reconstructs only C
• GPT-5.6-sol edge kernels: 64.5% correct, only 25.5% faster than tuned TVM
• Subspace Tuning freezes 99% of each weight matrix's spectrum: forgetting drops from 57 points to ~1
• Stopgrad flow maps converge exponentially even though the update is not a gradient

All paper explainers