Sokoban Eval: New 66-Room Benchmark Makes LLMs Play Classic Puzzle Game
banteg · x · 2026-11-14
Developer banteg released Sokoban Eval, a benchmark where language models play Jonah Ostroff's Heroes of Sokoban puzzle trilogy: 66 rooms across three parts with escalating mechanics (push/pull/swap, goblins and dragons, charm and wall-morphing). Models see the board image and text grid each turn and submit moves or full-room plans, replayed in the original PuzzleScript game, with active time counting inference and execution. Community members like kittenbeloved are already testing models like Fable 5.2 on the harder puzzles. The site also invites humans to beat the models, who currently hold the shortest-route record in every room despite shorter solutions existing.
More from Models
- rasbt maps Inkling 2 Small's 594B architecture built on attention residuals — rasbt · 2026-11-14
- Lab post-training researchers flood-likes post mocking on-policy distillation reality gap — teortaxesTex · 2026-11-14
- Mitsuhiko: Anthropic has no image model, yet its code that paints is shockingly good — mitsuhiko · 2026-11-14
- Rapid Gemini releases at the frontier spark super-exponential AGI optimism — iruletheworldmo · 2026-11-14
- Google exec confirms Gemini 4 pre-training run has started, its most ambitious yet — Independent-Wind4462 · 2026-11-14
- Masked decoding tames Gemma 3 12B text artifacts with LoRA width tags and RL — round · 2026-11-14