On a pure-planning LisanBench subtask, OpenAI models top the chart
scaling01 · x · 2026-10-25
scaling01 tested a derived LisanBench subtask with knowledge recall and graph construction removed: models get a 100-word graph and must find the longest simple path from a start word, approximating pure planning. OpenAI models dominate — without the edit-distance rule, GPT 6 Astra medium scores 56,140 (+659%), Gemini 4 Argon high 51,312 (+579%), GPT 6.1 Sol medium 49,887 (+731%), all three confirmed "loopers" at the top. The chart shows GPT-6.1-Sol reaching higher bank scores with far fewer output tokens across reasoning efforts than Muse Wonder Full and Opus 5.5.
Related event: OpenAI Models Sweep Top Spots on Pure Planning Benchmark(2 posts)→
More from Models
- CERN and Mistral partner to build the first foundation model for collider physics — burny_tech · 2026-10-25
- New ChatGPT Instant Has an 'Ackchyually' Problem, Users Complain — gregmushen · 2026-10-25
- Pirate Face's model weight "preservation" is just webseed redirects to Hugging Face's CDN — realmrfakename · 2026-10-25
- OpenAI's steep discounts flip Rippling's coding workloads from 57% Anthropic to 52% OpenAI — kimmonismus · 2026-10-25
- Training quirk: reverse-chronological data ordering worsens loss — kalomaze · 2026-10-25
- Matt Wolfe: open-source AI models like Kimi K3.1 Pro have caught up — Matt Wolfe · 2026-10-25