Marigold V2: single-step DiT depth estimation trainable on one 32GB consumer GPU
_akhaliq · x · 2026-09-10
Marigold V2 (to appear at SIGGRAPH Asia 2026) upgrades monocular depth estimation to a diffusion transformer: single-step inference, sharp edges, no OOM at 2K. Website, code, weights, and demo are all public.
- Built for small labs: starts from a big pretrained DiT (Qwen-Image-Edit-2509), 4-bit quantized with a rank-128 QLoRA, fine-tuned on a single 32GB consumer GPU — sensible depth in hours, done in days; no 80GB cards needed.
- Unconventional choices: V1 pushed depth maps through an image VAE; V2 aligns DiT features to ground-truth depth encoded via DINOv3 (iREPA-depth).
- Data insight: even the cleanest synthetic data (Hypersim) has broken depth at grass-blade scale, motivating SinkLoss, a loss tolerant of noisy ground truth without pixel-to-pixel correspondence.
- A 90-second paper walkthrough video is available.
Related event: Marigold V2: A Single-GPU DiT Depth Estimator Hits SIGGRAPH Asia 2026(11 posts)→
More from Research
- FrontierCS 2 launches on Kaggle: 48 tasks make agents beat published CS papers — MeganRisdal · 2026-10-27
- LightMTP: lightweight latent multi-token prediction boosts LLM planning in pretraining — gdm3000 · 2026-10-27
- DexPort: a handheld interface with a real robot hand for dexterous-hand data collection — pabbeel · 2026-10-27
- EMNLP study: AI literary translation reads fine but varies more, even within one book — tuvllms · 2026-10-27
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- Custom VQA benchmark under 48GB VRAM: Qwen3.8-Flash-Next leads, Qwen3.8-27B regressed — EmilPi · 2026-10-27