LLMs play a 2D platformer as a decision-making benchmark; 7.9B model only one to clear all 14 levels
Felladrin · reddit · 2026-10-17
Redditor Felladrin turned an LLM-plays-2D-platformer demo into an open benchmark: models read a text description of the game state and answer two multiple-choice questions per step (which objective to pursue, which key to press), so any model emitting per-option probabilities can play. Results are reproducible and feed a public leaderboard.
Leaderboard highlights:
- PurpleMIST-Flash-1.0 (7.9B) tops the board with 100% pass@1 and is the only model so far to clear all 14 levels across all 5 seeds
- Second-place Darwin-27B-ZTC (25.6B) manages just 40% pass@1 and 13.4/14 levels — bigger models don't win here
- Most other models score 0% pass@1, clearing anywhere from 13 down to 1 level
A 27B model takes 16 hours to complete a full trial (5 seeds, 100-death cap per level). Code and leaderboard are on GitHub.
More from Models
- Jeff Dean backs Elorian's Glimpse, a visual thinking model competitive with Astra on key benchmarks — JeffDean · 2026-10-17
- Meta's Muse Spark Open rated best US open-weight model on Signal65 PINNACLE — ryanshrout · 2026-10-17
- Liquid AI's d1-ultra blocks only dangerous commands in coding agent, no recalibration needed — JosephJacks_ · 2026-10-17
- Anthropic Releases Claude Opus 4.7, Its Most Capable Opus Yet for Long-Running Tasks — AaronBergman18 · 2026-10-17
- Researcher Quips: Text Summarization Isn't Solved, Yet We Eval Deep Reports — keunwoochoi · 2026-10-17
- Gemini Notebook to add step-by-step Slides and Video generation, hidden notice reveals — testingcatalog · 2026-10-17