Stanford's RT-J: 86M relational foundation model trained on 639 web-crawled databases
christopher · x · 2026-10-18
Jure Leskovec's group (Stanford + SAP) released RT-J, an 86M-parameter relational foundation model, along with THE JOIN, a corpus of 639 web-crawled relational databases with 6,023 LLM-generated forecasting tasks. Models and data are open on Hugging Face.
Per the arXiv paper (2610.17867, accepted to NeurIPS 2026): RT-J is pretrained on long contexts up to 8,192 database cells with in-context examples selected by a lightweight random-walk retriever, plus context tuning and ensembling for test-time compute. On RelBench in-context regression it needs 10x fewer examples on average to match strong feature-engineering pipelines (LLM-agent/RDBLearn featurizers, TabICLv2/LightGBM predictors).
More from Research
- Labs spend $10M+ of compute on math—why not on mechanistic interpretability? — Miles_Brundage · 2026-10-18
- Random layer-8 neuron matches specialized neuron at carrying AI personas — Sauers_ · 2026-10-18
- RECON paper: representation conditioning beats alignment, 25x faster than REPA — udmrzn · 2026-10-18
- New interpretability technique clusters DPO-induced failure modes from preference data — AkshatS07 · 2026-10-18
- Goodfire essay: neural networks are already clean—polysemanticity was misread — rajistics · 2026-10-18
- OpenAI releases 700+ AI-generated math papers, with claimed progress on Riemann hypothesis — PolarBearby · 2026-10-18