Custom Graph Models and PFNs Outperform LLMs on Tabular Data

Researchers and practitioners recently debated whether Large Language Models (LLMs) are suitable for handling tabular data. The core conclusion is that in many real-world scenarios, custom graph models or PFNs trained from scratch outperform LLMs that are forcibly adapted. Furthermore, some current academic review opinions are disconnected from practical validation capabilities.

Confirmed

Gael Varoquaux confirms that PFNs use an architecture natively supporting numerical inputs without stringifying numbers and are trained from scratch. In contrast, post-trained LLM solutions like TabuLa-8B fall behind on large tables. Varoquaux notes that they tested models with up to 8B and even 30B parameters, yielding suboptimal results while consuming heavy resources. He emphasizes that reliable evaluations of tabular learning require cross-validation across hundreds of tables, often containing thousands to hundreds of thousands of rows. Insufficient past evaluation is a major reason for the stagnation in this field. Additionally, @yoavgo and @roydanroy mentioned in their exchange that in some real-world business consulting, understanding the domain reveals that the most suitable solution is actually customized graph models (such as LDA variants popular a decade ago) rather than forcing the use of LLMs.

Unconfirmed

Discussions regarding NeurIPS review quality represent subjective venting and opinion clashes among researchers. Gael Varoquaux and others believe that many review comments appear reasonable on the surface but are fundamentally impossible to empirically falsify through experiments under standard research budgets. A reply jokingly suggested "post-training LLMs on joint distribution samples from graph models" to counter such reviews. This is community criticism and banter directed at the academic review mechanism, not a statement of objective fact.

Why It Matters

The capability boundaries of LLMs have always been an industry focus. This discussion specifically points out LLM shortcomings in tabular data tasks from the perspectives of technical architecture (native numbers vs. stringification) and evaluation methods (cross-table cross-validation). It serves as a reminder to the industry that amidst the LLM hype, the effectiveness and economic viability of traditional machine learning methods and targeted architectures should not be overlooked.

2026-07-26 ~ 2026-07-27 · 8 related posts

• [source] Consultant says the best ML fix was an old-school graphical model nobody wants to recommend — yoavgo · 2026-07-26
• For some ML problems, a bespoke graphical model may fit better than an LLM — roydanroy · 2026-07-26
• NeurIPS review jokes about impossible-to-falsify ideas get a TabulatPFN comeback — GaelVaroquaux · 2026-07-26
• NeurIPS reviewers are asking claims that are impossible to disprove on budget — roydanroy · 2026-07-26
• [source] PFNs use native numeric architectures, while post-trained LLMs lag on larger tables — GaelVaroquaux · 2026-07-26
• PFNs train from scratch for tabular data, while post-trained LLMs hit limits fast — roydanroy · 2026-07-26
• [source] Tabular learning needs cross-validation across hundreds of tables, researcher says — GaelVaroquaux · 2026-07-26
• Bespoke Models Still Beat LLMs on Real-World Tabular Workflows — roydanroy · 2026-07-27