Same Prompt, 21 Replays: Sonnet 5.5 Took the Closest Pokémon 13 Times, Opus 5.5 Never Did
VibeCodyH · reddit · 2026-09-29
In a Pokémon Red benchmark scored via RAM milestones, the author replayed Sonnet 5.5's starter pick 21 times with identical prompts: it chose the nearest Charmander 13 times, Squirtle 8, Bulbasaur 0 — while Opus 5.5 picked Squirtle 18 of 20. Crucially, the Charmander picks weren't blind: the model often knew Squirtle was better but took the "simpler path" around Oak. That cost it — Charmander's attacks were resisted by rock, it blacked out at turn 270, and grinded to a second-attempt win (470 turns, $10.49 vs Opus 5.5's 271 turns, $8.44; Fable 5.1 needed 554 turns and $99.76). Full logs at pokebench.tv.
More from Fun
- Vibe-coded in 2 days, viral game Skyfall Rush lets AI agents play via Spawn's API — TAbrodi · 2026-09-29
- Jensen Huang reportedly uses a Samsung Galaxy Fold phone — firstadopter · 2026-09-29
- Muse AI turns workout videos into frame-by-frame form scoring avatars — armand_ruiz · 2026-09-29
- AI logos converged on buttholes — and now their agents are converging on little poops — sterlingcrispin · 2026-09-29
- Dev wires Claude into a 2007 Nokia with 8MB RAM, reverse-engineers its firmware — iamaliveix · 2026-09-29
- Users Now Loop Subscriptions Between OpenAI and Anthropic as Models Commoditize — imjustnewatai · 2026-09-29