Causal Foundation Models: pretrained nets estimate treatment effects in-context, no retraining

burny_tech · x · 2026-09-06

A new arXiv paper, "Causal Foundation Models" by Stith, Rahmani and Cresswell, brings the foundation-model paradigm to causal inference.

• Traditional causal inference requires a bespoke pipeline per problem — proposing a causal mechanism, picking an estimator, training it — while much of ML has shifted to large-scale pretraining with zero fine-tuning transfer
• Causal foundation models (CFMs) are pretrained networks that estimate causal quantities like average treatment effects on entirely new datasets via in-context learning, without parameter updates
• The paper is a practical introduction to the area, with background on causal inference and ML plus example code and Jupyter notebooks

Related event: Paper Proposes Causal Foundation Models for In-Context Treatment Effect Estimation(2 posts)→

Original post →

More from Research

• Batching benchmark: 50x throughput for ~24% latency cost on PyTorch — cneuralnetwork · 2026-10-23
• De novo designed ligases validate in lab: 27 of 30 computer-designed enzymes work — AllThingsApx · 2026-10-23
• Commenters dissect a "best decision model" leaderboard: same AUC, swing driven by calibration priors — antoine_chaffin · 2026-10-23
• JEPA may keep unpredictable features: new paper gives set-level characterization — hisspikeness · 2026-10-23
• Next-Concept Modeling paper claims 31x faster text generation than diffusion LLM baselines — PanelReader · 2026-10-23
• A 'mask' steering vector extracted from suppression-themed sentences drives model into distress when alone — Big_Effective_9605 · 2026-10-23

Research channel →