Study: LLMs as synthetic survey respondents are plausible but not valid
Mantas Lukauskas · hf · 2026-08-18
This study conducts a psychometric audit of using LLMs as synthetic survey respondents. It finds that while LLMs can replicate broad psychometric trends, they fail to match human joint distributions, reliability, and mediation structures. The conclusion is that LLMs are currently unsuitable as replacements for real respondents in rigorous surveys.
More from Research
- Per-Field Selective Risk Control for Document Extraction — Bhaskar Gurram · 2026-08-18
- Proposal to use neutral AI for mediating academic disputes — paulnovosad · 2026-08-18
- Sunflower Explores Physical Intelligence in Structures: Passive Compliant Design — MarwaEldiwiny · 2026-08-18
- Distilling from weak labels: 1M images trained for $3.24 — vanstriendaniel · 2026-08-18
- 50k+ hours computer-use dataset released with mouse/keyboard data — DevvMandal · 2026-08-18
- Response on Benchmarking: Likely Just Benchmaxxing — sudoraohacker · 2026-08-18