Feeding SOTA models blueprint PDFs for walkable 3D renderings: improving but still off
mobileraj · x · 2026-09-23
Inspired by Simon Willison's pelican-on-a-bike test, the author gives each SOTA model a PDF blueprint and asks it to create a 3D rendering they can walk through. Findings: capability is improving — early SOTA models would time out, while current ones usually finish — but floor plans remain grossly inaccurate and often violate physics. A simple, replicable probe for tracking spatial understanding across model generations.
More from Models
- Unverified Early Impressions of Suspected Opus 5.5: Capable but 'Doesn't Feel Like an Opus' — repligate · 2026-09-23
- Artificial Analysis launches AA-Omniscience: all but 3 models hallucinate more than they answer right — geoffwolfe · 2026-09-23
- Anthropic launches Claude Opus 5.5: matches Fable 5.1 on most tasks at 40% lower cost — jyangballin · 2026-09-23
- AI researcher deletes poll on OpenAI math results, calling his framing misleading — ChrSzegedy · 2026-09-23
- Scale AI's Muse, built with Meta's Muse Spark 1.3, tops the App Store and PRBench — alexandr_wang · 2026-09-23
- Anthropic Opus 5.5 scores 93.3% on ARC-AGI-2 at 80% lower eval cost, ARC Prize confirms — burny_tech · 2026-09-23