Puffin-World: unified multimodal model generates physically grounded explorable 3D worlds from one image, all open-sourced
CSProfKGD · x · 2026-09-04
Puffin-World technical report released
Kang Liao, Chen Change Loy and colleagues at NTU released Puffin-World, a unified multimodal world model with paper, code, models and data all open-sourced.
Key points:
- From a single image or text prompt, it jointly models three native world states — physics (gravity field and latitude), geometry (depth), and appearance (image) — to generate camera-controlled views and reconstruct an explorable 3D world, without external offline modules
- A unified Omni-Camera representation grounds absolute camera properties in the real world, enabling physically consistent, visually stable viewpoint control
- A strategy for propagating physical dynamics across future frames supports closed-loop applications like mimic and self-calibrated world exploration
- Ships with the Puffin-16M dataset: 15M vision-language-camera triplets plus 1M trajectories with challenging motions
Paper: arXiv 2609.04196
More from Multimodal
- This stunning W51A star-forming region image was AI-processed from raw JWST captures — fofrAI · 2026-10-21
- Purify Before You Align: filtering locally predictable info improves diffusion alignment — volokuleshov · 2026-10-21
- Mistral AI launches Voxtral 3, a real-time voice agent stack in 50+ languages — The Decoder · 2026-10-21
- Full AI Virtual Influencer Character Sheet for 'JataJett': Turnarounds, Expressions and Action Modes in One Page — CurieuxExplorer · 2026-10-21
- AI satire video: 'Paris City Tours, 50% off — slightly different experience' — DavidmComfort · 2026-10-21
- AI-Generated Series 'The Sage' Hits Episode 8, Made Entirely with Invideo Agent — Kavanthekid · 2026-10-21