Causal Foundation Models: pretrained nets estimate treatment effects in-context, no retraining

burny_tech · x · 2026-09-06

A new arXiv paper, "Causal Foundation Models" by Stith, Rahmani and Cresswell, brings the foundation-model paradigm to causal inference.

• Traditional causal inference requires a bespoke pipeline per problem — proposing a causal mechanism, picking an estimator, training it — while much of ML has shifted to large-scale pretraining with zero fine-tuning transfer
• Causal foundation models (CFMs) are pretrained networks that estimate causal quantities like average treatment effects on entirely new datasets via in-context learning, without parameter updates
• The paper is a practical introduction to the area, with background on causal inference and ML plus example code and Jupyter notebooks

Related event: Paper Proposes Causal Foundation Models for In-Context Treatment Effect Estimation(2 posts)→

Original post →

More from Research

• AI could crack Navier-Stokes on its own — and add almost no value to math — NathanpmYoung · 2026-09-06
• Stanford cs336 lectures give a shoutout to NoPE research — xhluca · 2026-09-06
• Tiny 1.5B local agent stops being confidently wrong with source-tier verification, finds real bug — UzairArain554 · 2026-09-06
• MasonKamb: gradient descent is the 'original sin' behind LLM-human cognition divergences — _arohan_ · 2026-09-06
• Chris Potts' IPAM talk on interpretability and subliminal learning now available — ChrisGPotts · 2026-09-06
• Single-metric robustness claims for LLMs can mislead, multi-level arXiv study finds — burny_tech · 2026-09-06

Research channel →