Sokoban eval: Opus and Sol with a Python REPL outpace pure-thinking rivals
banteg · x · 2026-11-10
An update on banteg's live Sokoban eval: the screenshot shows Claude Opus 5.5, Claude Sonnet 5.5, GPT-6.1 Sol/Terra, and GPT-6 Astra racing in both pure-thinking and '+Python' tool modes over a 5-hour limit. The author's read: Sol with the right tools is faster than Astra — both Opus and Sol with a Python REPL have reached the final level of part two, while top-row models that can only reason in their heads are falling behind.
Related event: banteg Launches Sokoban Eval for LLM Planning(2 posts)→
More from Models
- Devs Report Claude Fable 5.2 Underperforms Opus 5.5 on Coding Tasks — seanwbren · 2026-11-10
- Hermes Agent Matches Claude Code on Benchmarks but Costs More per Task — EXM7777 · 2026-11-10
- DeepMind's Samuel Albanie: fable 5.2 is SOTA for clarity on technical topics — SamuelAlbanie · 2026-11-10
- Hands-on: Tencent's Hy4 matches or beats rivals across five agentic coding tests — CodeByPoonam · 2026-11-10
- Unverified rumor: Gemini 4 Carbon rumored to launch today as UGI board shows Argon tiers — Eissa_Cozorav · 2026-11-10
- Max/Ultra Subscriber Burns Through 5-Hour Quota in 1 Hour, Slams Claude Inference Capacity — theteknosaur · 2026-11-10