39 LLMs tested on Rubik's cubes with no tools: only 2 solve a 20-move scramble
JeremyCMorgan · x · 2026-11-06
Brian Briscoe built RubiksBench: give 39 LLMs a scrambled cube as text, no tools, one reply, temperature 0, scored by playing the returned moves on a simulator.
- Only 2 of 39 models reliably solve a 20-move scramble; most can't even solve a 6-move one
- One model repeated "R row?" 15,912 times until it ran out of tokens
- Design: moves in the standard 18-move DSL, cube state as a 2D unfolded cross; four rounds (3/6/10/20 moves) with promotion gates to avoid burning tokens on hopeless calls
- Via claude.ai or chatgpt.com nearly every model succeeds — because they write scripts calling solver libraries like kociemba — so the harness deliberately goes through OpenRouter with tools disabled to test raw capability
The benchmark highlights how weak LLMs' internal state tracking is on tasks they appear to ace with tools.
More from Models
- theshawwn: government employees stay quiet, so Claude's unintuitive behaviors go unreported — theshawwn · 2026-11-06
- Nemotron 3.5 Super + Surge RL matches GPT-5.6 Terra across 12 benchmarks — echen · 2026-11-06
- DeepSeek V4.1 Flash hits 600+ tok/s on Isoquant, 8.7x faster than Claude Opus 5.5 — ycombinator · 2026-11-06
- METR: Evidence on Model Capabilities Has Multiple Plausible Readings, Conclusions Remain Uncertain — morqon · 2026-11-06
- Ant's Ling-3.2-flash: 560B-Parameter Multimodal Model Claims to Beat Claude Opus 5 — realmrfakename · 2026-11-06
- OrcaCyber-Nano-1.0, a Qwen finetune, praised as frontier-adjacent on 24GB VRAM — JackMasterN1 · 2026-11-06