Researcher pushes back on LLM-simulated surveys: AI is a text sampler, not a model of human psychology
gerardsans · x · 2026-10-26
Data visualization researcher Gerard Sans pushes back on the Nature study shared by John Horton showing GPT-4 can predict social science experiment results: running surveys on a "compressed file of the Internet" is a meaningless premise. He argues research should stop casual anthropomorphism — LLMs are text samplers, not people, with no psychology, only data-compression artifacts as biases. In a follow-up he adds that frontier models are trained on low-quality internet data contaminated by fiction and marketing copy, making such survey data "dangerous": the wrong kind of data, riddled with unknown variables, mistaking textual fluency for scientific validity.
Related event: Debate Erupts Over Whether GPT-4 Can Predict Survey Experiment Outcomes(2 posts)→
More from AGI Musings
- Ben Thompson's 'The End of the Internet': when creation goes free, winners go small — tushaarmehtaa · 2026-10-26
- Revolut CEO: China's 9/9/6 work culture makes it unbeatable — only 2 of 140 startups survived — davidsenra · 2026-10-26
- AI Is Redistributing Technical Power Inside Companies, Not Just Creating New Roles — mdancho84 · 2026-10-26
- Two New Papers Quantify AI's Impact on India's Labour Market With PLFS Data — ravi_iitm · 2026-10-26
- AI thinking is to human thought what planes are to bird flight — js_craft_hq · 2026-10-26
- Stratechery: endgame for the open web, and a coming AI startup credit crunch — tushaarmehtaa · 2026-10-26