Block3D: Efficient Text-to-3D Generation via Block-Wise Diffusion
Bowen Cui, Weijie Wang, Zeyu Zhang, Yefei He, Mingda Lin, Haoyu Zhao, Yuanyu He, Donny Y. Chen, Feng Chen, Bohan Zhuang
cs.CV
2026-08-20
ZIP Lab turns Cube's token-wise 3D prior into block-wise denoising. On 100 held-out objects, mean time falls from 25.71s to 4.99s and Chamfer-L1 from 0.094 to 0.078.
Text-to-3D is stuck on a latency-versus-detail trade. Autoregressive mesh generators emit one shape token at a time, so long codes are slow and irreversible. Diffusion and flow models update many latents in parallel, then pay for that by rescanning the full state at every step. ZIP Lab at Zhejiang University, with Monash and Adelaide, moves the causal cut from single codes to contiguous blocks, aiming to keep geometry while cutting wall time.
The frozen Cube VQ stays: each mesh is 1024 codes from a 16384-entry book, and the mesh decoder is frozen. Text comes from frozen CLIP ViT-L/14 (77 tokens). Only Cube's 23-layer DualStream RoFormer prior is fine-tuned.
Those 1024 positions are split into blocks. Default block size is 64, so 16 blocks, generated left to right. The active block starts fully masked, attends bidirectionally inside itself, and reads the committed prefix causally. Future blocks are absent. Up to T=4 updates per block do two edits: fill high-confidence masks (M2T) and replace already-filled codes the model now prefers (T2T), with thresholds 0.95 and 0.9. A deterministic quota finishes every mask within four steps. Classifier-free guidance falls from 3.0 to 0.75. Guided logits propose a code; the conditional softmax decides whether to accept it, so CFG does not inflate the confidence score.
Training samples a mask stream or a substitution stream per example, with corruption rates in [0.45, 0.95]. One no-grad rollout step runs first; the loss then hits only positions that still disagree with the target. Prefixes are teacher-forced in training and model-generated at inference, so exposure bias remains, but only across block boundaries.
Evaluation is 100 objects held out from TRELLIS-500K with seed 42, excluded from a 300K fine-tune run of 35K steps. Geometry uses 8192 surface points. Latency on one A100 80GB covers condition encoding, code generation, and mesh decode.
| Method | CD-L1↓ | NC↑ | F@1%↑ | CLIP | Mean time |
| ShapeLLM-Omni | 0.229 | 0.490 | 0.089 | 19.08 | 36.89s |
| TRELLIS-text | 0.222 | 0.496 | 0.090 | 20.41 | 11.65s |
| Cube | 0.094 | 0.632 | 0.219 | 23.87 | 25.71s |
| Block3D | 0.078 | 0.668 | 0.309 | 23.24 | 4.99s |
Against the matched Cube fine-tune, mean time falls from 25.71s to 4.99s (5.15×). Median and P90 drop with it, and the standard deviation shrinks from 0.80 to 0.43. All three geometry scores beat Cube. CLIPScore slips from 23.87 to 23.24.
Block size is the main knob. B=32 takes 12.98s at F@1% 0.296; B=64 is the chosen point; B=96 is 3.62s with F@1% 0.279; B=256 is 2.15s with F@1% 0.103. Raising T from 4 to 20 costs 13.66s and does not beat four-step F@1%. Adding T2T lifts F@1% from 0.287 (M2T only) to 0.309.
This is Block Diffusion plus editable discrete denoising, wired onto a fixed-length 3D codebook, not a new geometric representation. For stacks already on Cube codes, swapping the prior cuts end-to-end time from tens of seconds to about five, with better paired geometry. The CLIP dip shows the speed win is not a free alignment win.
The schedule uses at most 80 batched forwards (160 logical branches with CFG), versus Cube's 1024 sequential token steps. That accounting is not a substitute for the measured 4.99s.
Committed blocks never reopen, so cross-block exposure bias remains. Blocks follow Cube's 1-D code order, not semantic parts, unlike part-wise denoisers. The test set is 100 objects with one sample per prompt and no diversity metric. Cube is a matched fine-tune; TRELLIS-text, ShapeLLM-Omni, and AR3D-R1 run official inference settings. CLIP is worse than Cube, so text-shape alignment is not along for the ride.