Psychometric audit of 47 LLMs: synthetic survey respondents still trail copula baselines
Mantas Lukauskas · hf · 2026-10-14
Testing 14 decision models from 11 providers and 47 generative LLMs as synthetic respondents on a Lithuanian organisational-psychology survey (n=263, 68 items, 100 real profiles), the best generative model MiniMax M3 hits PSS 0.813 — level with a Gaussian copula fit only on other respondents (0.824) and below an in-sample fit (0.927). Five of ten product lines regress beyond resampling noise in their newest release; 13 of 14 decision models score below gpt-5.4-mini with default readout. Probability vectors match human marginals and rank respondents decently, but as personal forecasts all decision models lose to other respondents' answer frequencies, and profile regression improves on those by only 1.0% — synthetic respondents should target sample-level fidelity.
More from Research
- NTU team has Claude write robot control code, hitting 70.24% success in sim — aigclink · 2026-10-14
- Six training strategies compared: can LLMs reason over graphs at scale? — burkov · 2026-10-14
- VisInteract: NeurIPS Spotlight reframes text-to-visualization as multi-turn intent retrieval — jiqizhixin · 2026-10-14
- WorldRecon reconstructs large-scale 3D scenes from unposed multi-view images, cutting Chamfer distance 43% — ZhejiangUniversity · 2026-10-14
- RSI-Nav: recursive self-improvement for navigation agents hits 68.6% SR on R2R-CE — OpenGVLab · 2026-10-14
- NUS proposes Jeva, a decision-native architecture with near-perfect null detection at thousands of candidates — NationalUniversityofSingapore · 2026-10-14