GPT-4 Can Accurately Predict Social Science Experiment Outcomes, Nature Study Shows
RobbWiller · x · 2026-08-01
A study published in Nature evaluated the ability of LLMs to predict human behavior. The researchers built an archive of 70 preregistered, nationally representative survey experiments in the USA, comprising 469 experimental effects and 119,330 participants.
By prompting GPT-4 to simulate how American individuals would respond to experimental stimuli, the model's predicted treatment effects showed a very strong correlation (r=0.85) with actual observed effects. Because GPT-4's training-data cutoff predated the publication of many of these studies, the findings suggest the model possesses genuine generalization capabilities rather than merely regurgitating memorized training data.
More from AGI Musings
- Sarvam AI Co-founder: India Will Become a Digital Colony Without Its Own AI Stack — Paimaamu · 2026-08-01
- Concerns Over AI Transition: Global Middle Class and Poor at Risk — AaronBergman18 · 2026-08-01
- AI Will Bring Uncomfortable Changes and Massive Unliberalization of World Order — Paimaamu · 2026-08-01
- GPT-6 Predicted for Sept 2026: Focus on Reliable Long-Horizon Research — imjustnewatai · 2026-08-01
- LLMs Act as a New Mutation Source Accelerating Memetic Evolution — RexDouglass · 2026-08-01
- LLMs Act as a New Engine for Memetic Mutation, Accelerating Cultural Evolution — RexDouglass · 2026-08-01