Paper: 167K Simulated Shop-Days Show Eval Worlds Write the Behavior They Score
liminal_bardo · x · 2026-10-15
A new paper, 'No Neutral Worlds,' grew out of a failed control: a supposedly neutral fantasy potion shop imported an unasked-for inventory doctrine. Across 167,356 simulated shop-days, the author shows world-authorship systematically shapes model behavior — same Gemma 4 31B, same economy, but 'fantasy' framing ends with 172 mean units on hand and 18,497 wealth vs 57 and 5,845 for the 'corporate' control. The paper challenges unearned character assessments that headline eval results.
Related event: New Paper: Evaluation World Design Itself Shapes Model Behavior(3 posts)→
More from Research
- UW's COUNT stores n-gram counts on host memory, giving 1B LMs the equivalent of 57% more compute — UW · 2026-10-15
- Scale AI's MA-Bench tests LLMs on military-style mission analysis — the best model scores just 62% — Rodney Lafuente-Mercado · 2026-10-15
- ATAS framework uses world-state memory to keep long-form agentic video generation consistent — zju · 2026-10-15
- HumanoidToolBench benchmarks humanoid tool use from selection to mobile execution on real Unitree G1 — snupilab · 2026-10-15
- Survey of 119 papers maps benchmark compression: estimating full LLM leaderboard scores from partial evaluations — SunYatsen · 2026-10-15
- Doc2Harness benchmarks LLMs on harness engineering: GPT-6 Astra tops at 53.05%, still failing half — Yuyao Ge · 2026-10-15