13 AI models vs. real policymakers on policy questions: 10 models beat the humans

ChenhaoTan · x · 2026-11-20

Building on interviews by Ben Olken, Alexa Weiss and remanadeem with current and former ministers on questions like whether cash transfers reduce work and health-insurance uptake, ChenhaoTan sent the same policy questions to 13 AI models. The median policymaker scored 86/100 on evidence alignment, but 10 of 13 models scored higher: Claude Fable 5.2 led at 98, followed by Kimi K3.1 Pro (97) and GLM 5.3 (96). Agreement-with-policymakers rankings differ (Muse Wonder 82 top, Gemini 4 Carbon 57 bottom). As public-sector workers increasingly consult AI, verifying answer quality becomes critical.

Original post →

More from Models

Models channel →