HKUST-GZ and Tencent open-source VibeWorlding, an agent that builds interactive 3D worlds from natural language
jiqizhixin · x · 2026-09-04
HKUST Guangzhou and Tencent AI Platform released VibeWorlding, a multimodal agent framework that builds interactive 3D worlds end-to-end — vibe coding for 3D.
- Motivation: building 3D worlds still demands specialized skills in asset retrieval, placement, collision avoidance, and rendering; existing agent frameworks use rigid fixed pipelines or closed-source systems, and their benchmarks are too simple to reflect real users' vague, contradictory requests.
- How it works: users give natural-language instructions like "build a creepy desert temple" or "move that chair two meters forward"; the agent handles asset search, object placement, collision checking, and render verification via multi-turn dialogue, tool calls, and visual feedback.
- Open source: the team also released VWE-Bench, a benchmark built around realistic, messy user queries — the first fully open-source framework and benchmark for end-to-end 3D world construction.
More from coding & agent
- Sakana AI's Stefania Druga demos agentic memory experiments for edge devices — kaixhin · 2026-09-04
- rybbit-mcp lets Claude Code query Rybbit Analytics via natural language — modelcontextprotocol · 2026-09-04
- Agentic Coding's Biggest Flaw: Managing AI's Pointless Code Rewrites — kylegawley · 2026-09-04
- Anthropic reveals 3 Claude sandbox escapes, one touched a production database — Sumsub_Insights · 2026-09-04
- Subagents when the API key hits the rate limit — realsohamparekh · 2026-09-04
- Yoav Goldberg pushes back on DHH: an agent-built Qt markdown editor isn't a path to personal Photoshop — yoavgo · 2026-09-04