Orbia benchmark: 27 video models fail to keep a consistent 3D world
CSProfKGD · x · 2026-10-18
Orbia: A new benchmark testing whether generated videos form a consistent 3D world
The Orbia team released a benchmark that tests whether video generation models and world models can preserve the input scene, retain newly generated content, and produce views that agree in 3D as the camera explores.
Scale and design
- 720 exploration cases (480 short + 240 minute-long)
- 6 trajectory protocols: Explore, Revisit, Return, etc.
- 6 evaluation dimensions covering input preservation, generated-content persistence, and route-wide 3D self-consistency
- 27 models evaluated, supporting camera, action, and text interfaces — including models without camera control
Key findings
- Strong visual quality and accurate camera control do not ensure 3D consistency
- Models can preserve the input yet lose newly generated content
- Local cross-view agreement can coexist with global geometric inconsistency
Each case starts at pose P0, passes a cross-view query, discovers and revisits a new region, then returns to the exact starting pose. Code, data, and the evaluation toolkit are open source.
More from Multimodal
- fal is so fast someone built a generative filesystem on top of it — gorkem · 2026-10-18
- Community Pins MOSS-Image SFT Vertical Banding to Layers 37-41, Ships De-Banding LoRA — MatlowAI · 2026-10-18
- Flux 3 Klein Gets Updated ComfyUI Support Following Community Criticism — ganrocks007 · 2026-10-18
- Recap: flow model + image model + planner GRU video generation architecture — pixlpa · 2026-10-18
- Krea 3's own site suggests it's an image editor, not a strong text-to-image model — LemonLinePL · 2026-10-18
- MOSS-Image comes to Forge Neo: readable English/Chinese text, 8-step generation — digitalhunters0 · 2026-10-18