SimulacraBench: NeurIPS 2026 competition to benchmark AI population simulation
sanmikoyejo · x · 2026-08-18
SimulacraBench is a NeurIPS 2026 competition designed to test how well AI models can predict survey responses and behavioral outcomes. It addresses data contamination by using three unpublished instruments from UNHCR, UNICEF, and the World Bank, covering 40,000 respondents across 14 countries with 0% overlap in the pretraining corpus. The starter kit is now live.
Related event: NeurIPS 2026 Launches SimulacraBench to Test AI Population Simulation(2 posts)→
More from Research
- Artificial Analysis Launches Search Index Benchmarking Agent Search APIs — ArtificialAnlys · 2026-08-19
- Artificial Analysis Launches Search API Index: Parallel, Exa Lead — ArtificialAnlys · 2026-08-19
- DataSmith auto agent outperforms model architecture tweaks using 200x fewer tokens via data interventions — josh_wills · 2026-08-19
- Pander Score evaluates sycophancy in AI models — eli_lifland · 2026-08-19
- MIT study: AI art often can't be traced to training data — nordicinst · 2026-08-19
- DynaRobotics podcast: scaling robot pretraining toward 1 million hours of data — chris_j_paxton · 2026-08-19