Pokemon Red benchmark: Opus 5.5 beats Brock for $8.44; Fable 5.1 needed $99.76
VibeCodyH · reddit · 2026-09-27
PokeBench drops models into a fresh Pokemon Red save, giving one screenshot per turn plus game state and on-screen text — no pathfinding, no map hints, 1,000 turns to beat Brock:
- Opus 5.5: beat Brock on turn 271 for $8.44 in 1h06m, never blacking out. The run was decided on turn 13, when it deliberately walked past the first ball: "pick Squirtle, since it's strong against Brock"
- Fable 5.1: grabbed the nearest ball and picked Charmander, never considering Brock. It was actually ahead entering the gym on turn 230, then got wiped by a Jr. Trainer's Sandshrew while its reasoning showed pure Tackle brute-forcing
- On turn 363 it said it would catch a Mankey to counter — but never went, ground in Viridian Forest instead, and finally won on turn 554: $99.76, 3h17m, three blackouts
Full turn logs with prompts and reasoning are published at pokebench.tv.
Related event: PokeBench tests LLMs on Pokemon Red; Opus 5.5 beats Brock for $8(2 posts)→
More from Fun
- One prompt: Claude Opus 5.5 creates entire alien-planet trailer, story to score — VoidStateKate · 2026-09-27
- Dev's tested AI music workflow: lyrics, Suno, ear-curation, then Ableton — ctjlewis · 2026-09-27
- User speculates OpenAI may reveal internal model 'bel' at DevDay — imjustnewatai · 2026-09-27
- Claude Opus 5.5 Chains Blender, Image-Gen and three.js Into Browser-Running 3D — burny_tech · 2026-09-27
- Puppy Kill Bench: most models refuse, GPT6-Luna just executes the kill tool — MetroidsSuffering · 2026-09-27
- Claude kills plan mode the same day Google announces launching it — dotey · 2026-09-27