Astra saturates blogger's spatial reasoning eval, prompting 'LLM vision is solved' claim
lukaszkaiser · x · 2026-09-06
Blogger spiceylemonade's self-built spatial reasoning benchmark had failed every new model tested over time. Astra answered every sample correctly and fully saturated the eval, leading the blogger to declare 'LLM vision is solved.' Reposts note spatial reasoning was one of the last areas where humans vastly outperformed computers. The maker of Astra is unverified.
Related event: GPT-6 Astra saturates spatial reasoning benchmarks(4 posts)→
More from Models
- Claude $200 plan delivers 45.3x API value, author denies Anthropic quota cut — RexDouglass · 2026-10-23
- User slams OpenAI's GPT-6.1 Sol: 20 hours lost, ignores human feedback — RexDouglass · 2026-10-23
- Dexmal's open-source VLA model DM0.5 lands in LeRobot, tops RoboColiseum — AdinaYakup · 2026-10-23
- Dev: all open-source audio models today are subpar, even representation ones — A_K_Nain · 2026-10-23
- Codex $200 subscriber burns through a week's quota in 16 hours, suspects silent usage change — zac_attack_ · 2026-10-23
- OpenAI's Tibo Sottiaux confirms a Codex reset is landing tomorrow amid quota complaints — jeppa35 · 2026-10-23