New paper: LLM-based simulators for policy design must generalize beyond frozen in-distribution data
soumitrashukla9 · x · 2026-11-05
A new paper by KehangZhu argues LLM simulation is only useful for market and policy design if it holds up after the rules change. Most evaluations stay in-distribution: the authors select what the simulator sees from past experiments, freeze it, and test on new people under new rules/designs to measure out-of-distribution generalization.
More from AGI Musings
- Beff Jezos: halting AI datacenters would be 'literally existential' mistake — beffjezos · 2026-11-05
- Evolution can't copy homework: why species improve slower than iPhones — louisvarge · 2026-11-05
- 1889 newspaper clipping: New York's 'crusade against the deadly electric wire' — daniel_mac8 · 2026-11-05
- LLM Era May Leave Only Three Software Roles: PMs, Factory Maintainers, LLM Adversaries — acostin · 2026-11-05
- 'I run everything through AI' is becoming an anti-signal — mobileraj · 2026-11-05
- Dan Grover on beating AI-induced purposelessness: go out-of-distribution — DanGrover · 2026-11-05