Direct3D-V3: latent-free diffusion transformers generating 3D directly in the native voxel domain
NJU · hf · 2026-11-20
Direct3D-V3 proposes a latent-free diffusion transformer framework that discards the VAE entirely and operates directly on raw SDF voxel grids during both training and sampling.
For geometric reliability:
- Implicit manifold regularization over neighboring voxels suppresses non-watertight surfaces
- Predicted SDF grids are converted to meshes via Marching Cubes so rendered normal losses supervise the model directly
- An auxiliary task predicts the conditioning image's camera parameters, improving image-geometry alignment
The latent-free design removes the reconstruction bottleneck and yields meshes with substantially richer local detail. Predicted cameras and native SDF output also enable generative reconstruction, interleaving diffusion with normal-guided refinement while the generative prior completes unseen regions. Authors are from Nanjing University, HKU, Oxford and DreamTech.
More from Multimodal
- Google ships 8 updates in 2 weeks: Gemini 4 Neon/Hydrogen/Carbon, Genie 4, Robotics 2.5 — GoogleAI · 2026-11-21
- CapCut plugin connects ChatGPT to auto-cut gameplay highlights, then human picks the payoff — iamfakhrealam · 2026-11-21
- Posterior Matching Distillation preserves teacher diversity in few-step generation — ArashVahdat · 2026-11-21
- Qwen3 TTS goes live on Together AI: 9 voices, plain-language control, 10 languages — togethercompute · 2026-11-21
- 3DGS vs Point Cloud comparison looks great; surface reconstruction for sim collisions next — jonstephens85 · 2026-11-21
- AI video showing 25+ celebrities 'before and after' plastic surgery goes viral — evisoft · 2026-11-21