Marigold V2: turning DiT generation models into SOTA depth estimators
Cohere · youtube · 2026-10-17
A Cohere Labs talk by Thiemo Wandel (Huawei Zurich) on Marigold V2, a set of techniques for repurposing modern DiT-based image generation and editing models into state-of-the-art monocular depth estimators.
The recipe targets single-step inference from pretrained multi-step flow-matching models, with quantization where needed—preserving capacity while staying cheap. The team analyzes naive-training artifacts and adopts a 2-stage fine-tuning protocol built around a novel Sinkhorn-based loss. Results: 16–26% AbsRel improvement over the previous best on KITTI and ETH3D, resolving fur, foliage, and hair-thin edges, plus SOTA when transferred to surface normals estimation and intrinsic image decomposition.
More from Multimodal
- Angle-control LoRA for FLUX.3 Klein image model lands on Hugging Face — Illustrious_Row_9971 · 2026-10-17
- fal Worlds put to the print-and-scan test, revealing texture and noise structure — adamho · 2026-10-17
- F3's biggest letdown: VAE is identical to F2, no retraining done — ZootAllures9111 · 2026-10-17
- Viral AI-generated night street portrait looks indistinguishable from a real photo — Div_pradeep · 2026-10-17
- F3's biggest letdown: it ships the exact same VAE as F2 with no retraining — returnity · 2026-10-17
- Surreal AI music video "In Plain Sight" with original music — Hellish_NDE · 2026-10-17