LisanBench Author Teases Old Paper: OpenAI Models Excel at Pure Planning Task
scaling01 · x · 2026-10-25
Eval blogger scaling01 says he "should really just publish the old LisanBench paper." He previously revealed that on a derived task stripping out knowledge recall and graph construction, OpenAI models remain extremely strong: given a 100-word graph, models must find the longest simple path from a starting word — a task closest to measuring pure planning ability. The tease suggests OpenAI models are underrated on certain reasoning dimensions.
Related event: OpenAI Models Sweep Top Spots on Pure Planning Benchmark(2 posts)→
More from Models
- KLD Is Misunderstood: Why It Can't Rank Quantized Models Reliably — Positive-Resource922 · 2026-10-25
- Reflection AI open-sources Beam: 500B-parameter model trained in 2 months with near-zero downtime — dl_weekly · 2026-10-25
- GLM finds trivial full RCE in a car head unit via debuggable system app and JDWP — ngxson · 2026-10-25
- Ai2 releases OlmoEarth v1.3: state-of-the-art 10m-resolution embeddings for the whole planet — Kyle_L_Wiggers · 2026-10-25
- ECI capability curve flattens as frontier model gains slow — quadriphasic · 2026-10-25
- Unsloth Flint Q3 GGUF flops on Flint Benchmark, best harness 71/100 — mesolimbic-bliss · 2026-10-25