13 AI models vs. real policymakers on policy questions: 10 models beat the humans
ChenhaoTan · x · 2026-11-20
Building on interviews by Ben Olken, Alexa Weiss and remanadeem with current and former ministers on questions like whether cash transfers reduce work and health-insurance uptake, ChenhaoTan sent the same policy questions to 13 AI models. The median policymaker scored 86/100 on evidence alignment, but 10 of 13 models scored higher: Claude Fable 5.2 led at 98, followed by Kimi K3.1 Pro (97) and GLM 5.3 (96). Agreement-with-policymakers rankings differ (Muse Wonder 82 top, Gemini 4 Carbon 57 bottom). As public-sector workers increasingly consult AI, verifying answer quality becomes critical.
More from Models
- Gemini 4 Neon one-shot a Shibuya runner game starring Google's Pegman — rseroter · 2026-11-20
- Professors built 'AI-proof' exam questions — GPT-5.4 still scored 83% — alex_verem · 2026-11-20
- User tests Opus 5.5 via Claude Pro: after 24h it beats ChatGPT subs on value — mariofilhoml · 2026-11-20
- Google AI declares a border configuration 'mathematically impossible' and proves it — wrongly — conitzer · 2026-11-20
- Frontier models can unmask authorship from a single text, no tools needed — MohitIyyer · 2026-11-20
- Redditor slams Google AI search: six months of updates made it 'cheaper, weaker, gaslighting' — GlitterHeart-Bliss · 2026-11-20