Alibaba's TaoTex diffusion model generates native 3D materials, beating TRELLIS.2
jiqizhixin · x · 2026-11-20
Alibaba researchers introduce TaoTex, a diffusion model for native 3D material generation, tackling the problem that flat 3D surfaces with readable text need texture cues geometry alone can't provide.
Key ideas
- Data construction agent: builds assets with high-frequency textures (text, patterns) to supplement public datasets — using an MLLM to write/run Blender scripts, render, review and refine, with IGM generating UV textures and MLLM checking quality and alignment.
- Architecture: fuses local and global image features (multi-level DINO features + learnable viewpoint embeddings for multi-image input); shifts training loss from latent space to pixels to reduce compression-related detail loss.
Results: on 30 texture-rich TexVerse objects with single-view conditioning, TaoTex reaches 25.15 dB albedo PSNR vs 21.92 dB for TRELLIS.2 (both using ground-truth geometry); four-view conditioning is evaluated separately in the paper.
More from Multimodal
- AI turns Mediterranean illustration into interactive 3D world; Three.js wall-crash demo costs $6.98 — Saboo_Shubham_ · 2026-11-20
- Kling team's Coda-DaVinci reimagines video generation via Code2Video agents — KlingTeam · 2026-11-20
- Tencent's WeMM-Video pushes open-source omni-modal video understanding — tencent · 2026-11-20
- Kling's EgoCaptioner uses 50k samples and GRPO to master egocentric video captioning — KlingTeam · 2026-11-20
- WM^MLLM: Princeton and NVIDIA turn multimodal LLMs into editable world models — OpenEnvisionLab · 2026-11-20
- Open-source repo offers 500+ AI-generated low-poly 3D landmarks for maps under CC0 — tomjohndesign · 2026-11-20