UrbanGround benchmark: LLMs' long-range city navigation drops to 0%
机器之心 · wechat · 2026-09-05
Researchers from SJTU, NUS, Meituan, CUHK and Oxford built UrbanGround, a navigable 3D city sandbox from real Hong Kong geodata, with 810 verified tasks testing 10 multimodal models.
- Visual recognition reaches 75.0–93.8% accuracy, but direction understanding only 23.3–58.3% (Gemini-3.1-Pro below random).
- Short-range navigation peaks at 75.0%; long-range collapses to 0–3.8% (GPT-5.5: 75%→0%).
- With road closures and pedestrians, compliance drops to 10.0–46.7% and collision rates hit 76.3–90.0%.
The takeaway: models "forget the road after a few steps" — each step's judgment fails to carry through. Paper, code and a playable app are public.
More from Multimodal
- AI video now pulls off orbiting, diving camera moves around a speeding bicycle — umesh_ai · 2026-10-23
- AI art showcase: gothic winged warrior "Azrael-42" with porcelain-crack armor — LudovicCreator · 2026-10-23
- Moody AI-reimagined Harry Potter portraits showcase stunning cinematic generation — azed_ai · 2026-10-23
- Claude Code + ComfyUI MCP workflow for client work: paid models or open weights? — newaccount47 · 2026-10-23
- AI-generated character sheet wows: melancholy traveling cat 'Finch' fully spec'd — azed_ai · 2026-10-23
- Viral prompt template turns AI image models into children's book character-sheet generators — azed_ai · 2026-10-23