LLMs still bad at spatial reasoning, but beating an evolutionary optimizer proves generalization

teortaxesTex · x · 2026-11-06

Discussion of a spatial reasoning eval report: frontier models (including the latest Opus 5.5) did poorly, but a buried detail shows they beat an evolutionary optimizer baseline — near-certain evidence of some generalization, though unsurprising to anyone who knows LLMs also outperform evolutionary optimizers on oddball OOD coding tasks. The commenter also flags that the tool interface given to models was odd: one tool sets arbitrary finger poses (capped at 45° per-joint displacement from current pose), another moves the hand.

Original post →

LLMs still bad at spatial reasoning, but beating an evolutionary optimizer proves generalization

More from Models