Kaininja: Extending Native 3D Generators to the Part Level
Ruihan Yu, Lian Fu, Muyao Niu, Zheng-hui Huang, Yu-Ju Tsai, Sho Kuno, Fengbo Lan, Yonghao Yu, Erwin Wu, Ming-Hsuan Yang, Kaipeng Zhang, Zhixiang Wang
cs.GR, cs.AI, cs.CV
2026-09-14
KaiNinja packs TRELLIS.2 into dual-volume O-Voxels (~24s/image, no segmenter); Chamfer drops 40% vs the best cascade and beats the same backbone fine-tuned whole.
Native 3D generators such as TRELLIS.2 already turn one image into a high-fidelity mesh with materials. The mesh is one welded object. Downstream work wants parts: retexture one, rig one, simulate one, drop one into another scene. Segment-then-regenerate is slow and trapped by the segmenter. Building N into the network makes compute scale with N and imposes a cap.
The representation is the harder block. An O-Voxel stores one sheet of surface per cell. At a part contact, two nearly parallel faces fall in the same voxel, the quadratic fit collapses them to one vertex, and no resolution saves it.
KaiNinja moves PartPacker's dual-volume idea onto TRELLIS.2's own sparse grid. The part-contact graph is two-colored. Parts that do not touch share a volume, so any N fits in two streams. Parts are connected components of each volume. The model never predicts N, and the pipeline has no mask and no segmenter.
The two-stage cascade inherits the prior in different ways. The layout flow (coarse occupancy) copies the 30-block Transformer per stream and inserts zero-initialized cross-volume attention after blocks 6, 12, 18, 24, and 29. Training fits each stream alone, merges them, warms up only the cross-volume blocks, then fine-tunes jointly with an overlap penalty so the streams do not claim the same voxel. The refinement flow (fine geometry and appearance) keeps the released backbone: 20 blocks attend inside one volume, 10 attend across both, with a zero-init volume embedding for identity. The VAE stays frozen and the material stack is reused.
Training uses 19,132 objects from four sources: Articraft-10K (articulated assets written as programs by a language model, so part labels have no annotator noise), PartNeXt, TRELLIS, and Fusion360. The authors believe this is the first 3D generator trained on agent-authored assets. At inference the two volumes decode to meshes, with a cheap merge of duplicated occupancy and a relabel of fragments, still without a segmenter. At 512 resolution the run is about 24 seconds per object on one H100.
On 986 held-out objects that every method completes, KaiNinja leads all seven metrics.
| Method | CDW ↓ | [email protected] | [email protected] |
| Hunyuan3D-2.1 + X-Part | 0.0314 | 0.814 | 0.597 |
| TRELLIS.2@1024 + X-Part | 0.0350 | 0.780 | 0.559 |
| PartPacker | 0.0389 | 0.749 | 0.492 |
| KaiNinja | 0.0186 | 0.919 | 0.692 |
Against the strongest cascade, whole-object Chamfer is 40% lower and strict part F-score is about 0.095 higher (roughly 16% relative). Generate-then-segment costs an order of magnitude more GPU-seconds because the segmenter occupies four GPUs. The same backbone, same corpus, fine-tuned as a whole object (TRELLIS.2@512) reaches 0.830 [email protected] with a 1.8% fail rate; KaiNinja reaches 0.919 and 0.7%. The intro quotes a 38% Chamfer cut versus that fine-tune. Going to parts did not tax the whole object. Packing looks like a better representation of the same shapes.
Ablations: fragment relabel cuts the part count from 14.06 to 5.43 (ground-truth mean 5.85). The overlap penalty mainly reduces cross-volume copies at layout time and barely moves final-mesh numbers.
If the goal is parts from an existing native generator, hanging a segmenter and N forwards on the outside is the wrong default. The cost here is a constant two-stream overhead, N is uncapped, and open surfaces plus PBR survive. For games, CAD, and rigging, 24 seconds per image with detachable parts is close to usable. The surprise is the whole-object gain: generating into two packed volumes beats fine-tuning the same backbone as a single volume. That representation claim is more worth tracking than another SOTA row.
Dual volumes are inherited from PartPacker. When the contact graph is not bipartite, parts must be merged until it two-colors, so densely interlocking objects undersegment. Three failures recur: almost the whole object packed into one stream, one stream empty (9 of 1000 test objects), or both streams emitting near-duplicate wholes. Thin shells sometimes dodge the merge. The decomposition varies with the seed. The Stage-2 decoder drops faces, so the release still runs geometric cleanup. The PartPacker comparison also uses a different corpus, so the gap cannot be blamed on O-Voxel alone. Explicit control of part granularity, and learning to reassemble, are left open.