Puffin-World: open-source unified multimodal world model with native 3D states
ccloy · x · 2026-09-03
Researchers including Kang Liao, Xiao-Ming Wu and Chen Change Loy released Puffin-World, a unified multimodal world model that perceives, simulates, generates and reconstructs 3D worlds in one framework.
Instead of only producing plausible pixels, it represents scenes via three native world states:
- Physics: gravity field and latitude anchoring observations to the real world
- Geometry: depth capturing 3D structure
- Appearance: images and sequences
Its Omni-Camera representation enables camera-to-world understanding, camera-controllable text-to-image generation, image/text-to-3D world generation, challenging camera trajectories, native geometry prediction and 3D reconstruction without external offline modules. Training scales across the Puffin-16M dataset with diverse camera setups.
Code, models and data are open-sourced.
More from Multimodal
- Jason Silva Made a 25-Minute Psychedelic AI Film in Five Days by Describing It Out Loud — lmoroney · 2026-10-21
- Ex-Brain Games host made a 25-min AI film in 5 days using 'vibe directing' in iMessage — lmoroney · 2026-10-21
- Motion Launches a General Video Agent That Edits Footage and Mimics Any Reference Style — mhdfaran · 2026-10-21
- LiFT Loops a Shared DiT Core: FID 3.34 Lower With 60% Fewer Parameters on ImageNet — burny_tech · 2026-10-21
- Ant open-sources 5 Ming-3.0 speech models spanning TTS, audio understanding and ASR — Med1_Ai · 2026-10-21
- What keeps AI videos watchable: hooks, consistency and a payoff — aftahi_ai · 2026-10-21