Axolotl3D: a Unified Framework for Faithful 3D Shape Completion
Anita Hu, Maria Shugrina
ECCV 2026
cs.CV
2026-07-23
NVIDIA's Axolotl3D unifies occluded completion, multi-view reconstruction, and editing as constrained 3D completion, using a point-cloud anchor to reach SOTA on Toys4K and OmniObject3D.
Image-to-3D diffusion models like Hunyuan3D 2.1 and Trellis generate clean geometry from a single image. They share one assumption: the input is one complete, unoccluded image. Real capture rarely works that way. You get sparse views, partial occlusion, or a need to edit geometry on an existing mesh. The field has handled each case in isolation: Amodal3R recovers occluded geometry, ReconViaGen enforces cross-view consistency, Instant3dit and PrEditor3D handle editing. None of them compose.
Axolotl3D's claim is that all three are the same problem. Completing the occluded parts, fusing multiple views, and editing a region of an existing shape all reduce to building a complete, self-consistent 3D object from incomplete observation under geometric constraints. Pull out that common structure and one framework covers them.
The model rides on Hunyuan3D 2.1, whose shape pipeline is a shape autoencoder (ShapeVAE) plus an image-conditioned flow-based diffusion transformer (DiT). Axolotl3D swaps the DiT's single-image cross-attention for a multi-modal variant that ingests four input types at once: up to six images, a per-view visibility mask, camera parameters, and a partial point cloud.
Each input has a job. DINOv2 encodes image features. Camera parameters go in as Plücker embeddings, which encode each pixel's camera ray direction and get added to the image features, so multiple views can be aligned in one shared 3D coordinate frame. The visibility mask is patchified into 14×14 blocks: fully unoccluded patches are 1, occluded patches are 0, and background counts as unoccluded to impose a free-space constraint. The point cloud is the geometric anchor, encoded to fixed-length latents by VecSetX, and it is what pushes the model to stay faithful to geometry in known regions.
At fusion time each modality passes through its own gated FFN into a shared 1024-channel space, plus a learnable per-modality tag to separate visual from geometric tokens. The decisive piece is mask-biased cross-attention: occluded visual tokens get an attention bias of negative infinity and contribute nothing, unoccluded tokens get 0, and point tokens are unmasked. The model can only draw evidence from unoccluded regions.
Training uses synthetic augmentation over 407k meshes (TRELLIS-500K), sampling four regimes with equal probability: sparse views with occlusion, large-area point dropout, editing, and clean single/multi-view. This augmentation is what lets completion and editing live in one model. The model fine-tunes from Hunyuan3D 2.1 weights for 500k steps on 8 A100s.
Evaluation runs on Toys4K (4k synthetic objects) and OmniObject3D (6k scanned objects), split into single-view and multi-view, each with and without occlusion. Single-view with occlusion is the hardest setting and where Axolotl3D's lead is largest:
| Method | F-score↑ | vIoU↑ | CD↓×10 |
| Amodal3R | 0.6877 | 0.1886 | 0.6144 |
| ShapeR (concurrent) | 0.7453 | 0.2285 | 0.4622 |
| Axolotl3D | 0.9046 | 0.3265 | 0.2402 |
Switch to six views with occlusion and F-score climbs to 0.9689, against 0.9190 for the concurrent ShapeR and 0.8920 for Hy3D-Omni. Standard deviation is lower across every setting, so output is more consistent.
On robustness: replacing ground-truth points with predicted monocular depth (no occlusion) still yields 0.7616 F-score, above SAM3D at 0.7062 and ShapeR at 0.5626. Add camera perturbation (10° rotation, 10% translation, ±5% focal length) and F-score barely moves, from 0.7616 to 0.7620. The authors credit the Plücker camera embedding.
The ablations map the design tradeoffs. Remove the point condition and F-score falls from 0.9036 to 0.8074, the largest single drop, confirming the point cloud is what holds faithful completion together. Remove the visibility mask and the drop is small with barely visible visual change; it acts more as a safeguard against interference. Remove explicit camera parameters and multi-view metrics barely move (the network infers relative poses from visual cues), but single-view degrades more, because without camera info the 3D points cannot be aligned with image features.
For anyone in 3D reconstruction or content creation, the value is not having to wire up separate pipelines for occlusion completion, multi-view reconstruction, and editing. One model, one set of weights handles all of them. The point-cloud anchor keeps the model true to input geometry in known regions, so geometric signal coming in from scans or monocular depth estimates gets used rather than overwritten by the generative prior. The paper shows two downstream paths: estimate points and camera from a single image with MapAnything, then generate; or edit by inpainting one view with Stable Diffusion Inpainting while keeping the points of unedited regions for a geometry-consistent edit. These are not benchmark numbers, but they show the model plugs into real pipelines.
The caveat: all of this assumes the input points and cameras are reasonably clean. The authors flag this as future work.
The appendix is candid about this. Training uses fixed camera distance and intrinsics, so the object must be fully in frame; robustness to wild camera viewpoints is untested. Extremely thin structures sometimes fail to connect, and fine surface detail can be smoothed or thickened. The model is trained to follow input points faithfully and has not been trained to handle noise, so depth-discontinuity artifacts, Gaussian-splat outliers, and sparse sharp SfM edges all hurt it. Points are downsampled to 8192 via furthest-point sampling, and the model is not trained for very sparse or sharp-edge inputs. It generates shape only, not texture.
One more concern: against the concurrent ShapeR, multi-view results are close (0.9689 vs 0.9190). Axolotl3D's clear edge is in single-view and in consistency. If the use case is tidy multi-view reconstruction, the payoff of unification deserves a second look.