NAVER's IDeaL: distill four vision teachers into one student with zero real images

mbsariyildiz · x · 2026-09-01

NAVER LABS Europe's ECCV 2026 paper IDeaL asks: in multi-teacher distillation (e.g. UNIC combining DINO, iBOT, DeiT-3, dBOT), how far can you get when the teachers' training data is private, licensed or gone? Answer: surprisingly far with zero real images.

Method: treat pixels as learnable parameters and optimize Dead Leaves structured noise through the frozen teachers — the teachers themselves know what makes an image informative — yielding IDeaL samples that look unreal but are maximally informative. Two decorrelation losses drive optimization: patch-level (push pairwise cosine similarities of patch representations off-diagonal, giving self-attention work to do) and image-level (minimize pairwise CLS embedding similarity across the batch for global diversity), with no labels, dataset statistics, or privileged info.

Results (UNIC protocol: ImageNet classification, transfer, segmentation, depth): IDeaL beats Dead Leaves in every setting; students beat the weakest teacher on 3/4 tasks even with only 1K samples; at 1K images, IDeaL matches or beats a 1K ImageNet subset on classification, closing most of the gap with real-image distillation.

Original post →

NAVER's IDeaL: distill four vision teachers into one student with zero real images

More from Research