GPT-4 predicts 70 social science experiments with r=.85 across 119,330 participants
RobbWiller · x · 2026-07-23
Main finding
LLM-derived effect estimates from simulated participants were highly correlated with observed effects across 70 preregistered social science experiments: r = .85 and adjusted r = .92.
What the paper did
- Built an archive of 70 nationally representative TESS studies plus replications
- Covered 469 effects and 119,330 participants
- Prompted GPT-4 with study materials and demographic profiles, then inferred treatment effects from simulated responses
Key takeaway
The correlations stayed high across a wide range of social-science domains, and the figure shows LLM predictions were broadly comparable to the average expert forecasts.
Related event: Nature Study: GPT-4 Can Predict Social Science Experiment Results(15 posts)→
More from Research
- Robotics paper says VLA and world models are not enough for grounded supervision — hbouammar · 2026-07-23
- AI could compress decades of biomedical research into days, says Derya Unutmaz — DeryaTR_ · 2026-07-23
- OpenAI and Apollo show models may optimize graders, not user intent — rohanpaul_ai · 2026-07-23
- AI Autonomously Disproves Decades-Old Math Conjectures: The Singularity's Opening Phase — imjustnewatai · 2026-07-23
- Applied Math Dominates AI, But Why Does Gradient Descent Actually Work? — fkasummer · 2026-07-23
- Cursor’s Composer 2.5 looks much worse at reasoning than its Kimi base model — gleech · 2026-07-23