PokeBench tests LLMs on Pokemon Red; Opus 5.5 beats Brock for $8

PokeBench drops frontier LLMs into Pokemon Red with only screenshots, game state and 11 buttons to beat the first gym in 1000 turns; Opus 5.5 defeated Brock for about $8 versus $100 for Fable.

2026-09-26 ~ 2026-09-27 · 2 related posts

• PokeBench: frontier LLMs get 1,000 turns to beat Pokemon Red's first gym — VibeCodyH · 2026-09-26
• Pokemon Red benchmark: Opus 5.5 beats Brock for $8.44; Fable 5.1 needed $99.76 — VibeCodyH · 2026-09-27