JLD: perceptual distance from a frozen encoder's Jacobian, fitted in 35s from 100 images, beats LPIPS and DISTS
Shreshth Saini · hf · 2026-10-07
Researchers introduce Jacobian Lens Distance (JLD), a perceptual image metric derived from a frozen vision encoder rather than human labels.
Motivation: pixel error ignores human perception, while human-fitted metrics are tied to fixed data and resolution—e.g., DISTS correlation on TID2013 drops from 0.815 to 0.717 when resolution doubles.
Method: JLD uses the encoder Jacobian to find directions in early patch-feature space that most strongly affect the encoder output, forming a fixed metric tensor E[J^T J], the 'Jacobian lens'. It is fitted once from 100 unlabeled images in 35 seconds.
Results
- SOTA across four perceptual databases, consistently beating LPIPS, DISTS, PieAPP, and DreamSim.
- Robust to resolution changes: lens-term correlation only drops from 0.850 to 0.845 when resolution doubles on TID2013.
- JLD-fast is 4x faster than LPIPS-VGG with mean correlation 0.911.
- Extends naturally to video: 0.786 correlation on Waterloo IVC 4K vs 0.611 for VMAF.
More from Multimodal
- Blender's The City Generator brings Houdini-style procedural 3D city building to everyone — bilawalsidhu · 2026-10-07
- MiniCPM-V-4.7-35B-A3B quietly appears on Hugging Face without a model card — BreakfastFriendly728 · 2026-10-07
- E-commerce product showcase animation built with Opus 5.5 + Higgsfield — Tegadesigns · 2026-10-07
- Gemini Omni 1.1 Can Be Surprisingly Cohesive When Used Right, Demo Shows — Cold_Solder_ · 2026-10-07
- EmbeddingGemma 2 Demo: One Search Across Images, Video and Audio in Browser — tomaarsen · 2026-10-07
- Creator builds 4-minute AI sci-fi trailer MORO with Agent Two and AI-assisted sound design — LudovicCreator · 2026-10-07