Study: 9 LLMs can write neural game agents, but they act like hand-coded programs

burkov · x · 2026-11-17

A study tests whether LLMs can turn a game's known rules into weights for a small neural agent that succeeds on held-out levels without further LLM calls. The result is qualified: across 9 models and 20 private game families with held-out levels, styles, and starting states, models could produce working neural agents—but the agents behaved more like hand-coded programs than learned networks, and generally underperformed Python players written by the same models. The direction could in principle distill reasoning into compact trainable systems, but current weights lack signatures of learning.

Original post →

More from Models

Models channel →