PlaySuite benchmarks 14 open models across 5,734 video games
cgmsnoek · x · 2026-10-12
Researchers introduce PlaySuite, a large-scale benchmark for interactive visual intelligence: AI agents play 5,734 video games on a supercomputer, testing 14 open models in thousands of unfamiliar visual worlds instead of static benchmarks. The thread covers how they built it and what they learned about current models' visual reasoning and long-horizon interaction weaknesses.
More from Models
- Byteification retrofits token-based LLMs to work at byte level, fixing letter-counting failures — PontiEdoardo · 2026-10-12
- Same Model, Different Harness: OpenCode Refuses Far More Than Codex in Side-by-Side Tests — Current_Balance6692 · 2026-10-12
- User claims GPT's $20 tier may be gone, pricing change suspected — sujingshen · 2026-10-12
- METR's AI time-horizon chart hasn't updated since May, leaving frontier model gains unmeasured — haider1 · 2026-10-12
- AWS open-sources Strands Decider 2B, a laptop-runnable decision model with no text generation — dl_weekly · 2026-10-12
- Sonnet 5.5 High vs Opus 5.5 Low: task-level cost data to pick your coding model — Headfirst-Into-Ditch · 2026-10-12