No underwater IQA metric matches human perception: 73 metrics tested, best kappa just 0.23
DelhiTechnologicalUniversity · hf · 2026-11-16
A DTU team introduces TrustUIQA, a benchmark testing whether no-reference image quality metrics agree with human judgment on underwater image enhancement.
- Scope: 73 metrics — handcrafted underwater (e.g. UCIQE), learned underwater, vision-language, and general-purpose — evaluated against five human references, including a new paired-comparison study
- Finding: no metric consistently matches human preferences; the best worst-case Cohen's kappa is only 0.23 (Q-Align), versus a median inter-rater kappa of 0.41
- Failure modes: widely used UCIQE often favors edits people reject; on a Nano Banana Pro replica chain that should leave images unchanged, only 22 of 73 metrics decrease monotonically while noise grows 2.5x
- Release: benchmark, code, and dataset are open on GitHub and Hugging Face
More from Multimodal
- V-JEPA-LM: counterfactual supervision teaches LMs to read video dynamics with 20x less data — Keuntae Kim · 2026-11-16
- Midjourney sref codes render knit and embroidered Adidas-style retro sportswear — sergeantsref · 2026-11-16
- TetraCoder: adaptive tetrahedral encoding beats Instant-NGP PSNR with fewer parameters — ssh4net · 2026-11-16
- Open-source ComfyUI node enhances MiniMax H3 image conditioning for better texture and detail — Away_Exam_4586 · 2026-11-16
- Combining LTX Audio Nodes with MiniMax H3 for Lip-Synced Video in ComfyUI — CQDSN · 2026-11-16
- One character, two styles: AI generates matching 2D illustration and 3D render — azed_ai · 2026-11-16