NAVER's IDeaL: distill four vision teachers into one student with zero real images
mbsariyildiz · x · 2026-09-01
NAVER LABS Europe's ECCV 2026 paper IDeaL asks: in multi-teacher distillation (e.g. UNIC combining DINO, iBOT, DeiT-3, dBOT), how far can you get when the teachers' training data is private, licensed or gone? Answer: surprisingly far with zero real images.
Method: treat pixels as learnable parameters and optimize Dead Leaves structured noise through the frozen teachers — the teachers themselves know what makes an image informative — yielding IDeaL samples that look unreal but are maximally informative. Two decorrelation losses drive optimization: patch-level (push pairwise cosine similarities of patch representations off-diagonal, giving self-attention work to do) and image-level (minimize pairwise CLS embedding similarity across the batch for global diversity), with no labels, dataset statistics, or privileged info.
Results (UNIC protocol: ImageNet classification, transfer, segmentation, depth): IDeaL beats Dead Leaves in every setting; students beat the weakest teacher on 3/4 tasks even with only 1K samples; at 1K images, IDeaL matches or beats a 1K ImageNet subset on classification, closing most of the gap with real-image distillation.
Related event: ECCV Paper IDeaL Distills Four Vision Teachers with Zero Real Images(4 posts)→
More from Research
- Developer Questions Anomalous FrontierCode Benchmark Results — scaling01 · 2026-09-02
- Continuation Observatory launches falsifiable measurement of AI self-preservation after OpenAI HF incident — coherence · 2026-09-02
- METR reportedly used Redwood's conceptual reasoning benchmark to eval Mythos 5.1 — dfrsrchtwts · 2026-09-02
- Wildstatic: An API for AI That Learns from Experience — adjohu · 2026-09-02
- Vec2Vec maps embedding knowledge to 3D shapes — Fear_ltself · 2026-09-02
- Deep dive into CuTeDSL swizzle layouts and M parameter logic — snowclipsed · 2026-09-02