PFNs use native numeric architectures, while post-trained LLMs lag on larger tables
GaelVaroquaux · x · 2026-07-26
Gael Varoquaux explains that PFNs do not post-train LLMs. Instead, they use architectures that natively handle numbers without tokenizing them, and they train from scratch.
He contrasts this with papers like TabuLa-8B, saying that post-trained LLMs are only toy-scale solutions that work on very small datasets and fall behind once the number of rows grows.
Related event: NeurIPS Review Controversies and the Edge of Native Numerical PFNs(2 posts)→
More from Research
- New book explores how knowledge graphs and LLMs can build connected AI systems — blaizedsouza · 2026-07-26
- Paper argues every microsecond matters for GPU collective latency — TheZachMueller · 2026-07-26
- DAIR-AI’s prompt engineering guide collects papers, notebooks, and lessons on RAG and agents — blaizedsouza · 2026-07-26
- Graph RAG replaces top-k chunk retrieval with graph traversal, and claims 32x memory savings — blaizedsouza · 2026-07-26
- A robotic elephant-trunk gripper uses an internal camera to sense touch — rvp · 2026-07-26
- Reverse-engineered SolidWorks files reveal operation histories that could train CAD LLMs — yacineMTB · 2026-07-26