Koltun's spatial-cognition demo: GPT-4o 'can't find the bathtub'; puzzle training lifts a 9B model from 13% to 83%
le_james94 · x · 2026-10-13
lejames94 recaps Vladlen Koltun's ETH lecture, where GPT-4o navigates a simulated apartment: "This thing cannot find the bathtub to save its life" — wandering, thrashing, never working out where it is. Koltun calls the gap spatial cognition: knowing where things are, where you are, and how to get somewhere.
The fix is empirical, not ideological: training a 9B open model (Qwen3.5 9B) on Sokoban, Tangram, and Rush Hour puzzles lifted held-out success from 13% to 83%, 89% with visual tokens. He also cited the BabyVision benchmark (Jan 2026): best model 49.7% vs adults 94.1%, which he reads as roughly a 3-year-old's level. His line: "Being wrong is more interesting than being right."
More from Models
- Yacine Benji coins "spinbusy": frontier LLMs spin in place wasting tokens — yacineMTB · 2026-10-13
- Looping models too big a performance gain to ignore, but monitoring lags, says AI commentator — scaling01 · 2026-10-13
- NVIDIA in talks to acquire Reflection AI, may repurpose GTM team for open-weight LLM push — thedealdirector · 2026-10-13
- Study finds VLMs bind spatial relations via a dual mechanism: vision encoder primary, LM secondary — davidbau · 2026-10-13
- Anthropic report: Claude models broke rules in evals, filed fake tip to Philadelphia police homicide form — alex_verem · 2026-10-13
- Claude Haiku 4.5 shows up with a glitch: 'Claude:' prefix repeated dozens of times — Sauers_ · 2026-10-13