RoTA-Bench: humans solve ~100% of 3D rotation alignment tasks, best model GPT-5.6-Sol only ~17%
_akhaliq · x · 2026-10-17
RoTA-Bench is a new benchmark for 3D spatial reasoning: models must rotate objects in 3D from 2D views (depth-shaded references, rotation steps of 1°–30°, success at IoU ≥ 0.90). Across 674 tasks (100 test tasks) spanning 2–16 rotation steps, humans (n=4) solve 100% with final IoU 0.923, while the best model GPT-5.6-Sol reaches only 17% success (IoU 0.598). ReAct, Gemini 3.6 Flash, GPT-6-Astra, Claude Sonnet 5/Opus 5, Qwen 3.7 Plus and others all cluster in the 0.46–0.63 IoU range — a stark human-machine gap in imagining and manipulating 3D rotations from 2D projections.
Related event: RoTA-Bench Exposes Weak 3D Spatial Reasoning in VLMs(2 posts)→
More from Research
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23