DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation
Zhuchenyang Liu, Ziyi Wang, Yao Zhang, Yu Xiao
cs.IR, cs.CL, cs.CV
2026-08-11
DistilVDR distills an 8B visual document retriever into a 524M single-vector model. HiRes reaches 61.74 NDCG@5, 86.9% of the teacher, at a 15.6x smaller index.
Visual document retrieval (VDR) answers a text query by finding the most relevant page among a pile of document images. The field is dominated by 2B to 8B parameter models (single-vector models like Qwen3-VL-Embedding-8B, multi-vector ones like Tomoro-ColQwen3-8B). Indexing a million-document corpus costs tens of GPU hours, and serving is expensive. The two existing compression routes both fall short: training a smaller multi-vector encoder from scratch inflates index size 16x and scoring latency 100x, while distilling only the query side still drags along the huge document encoder. Neither yields a compact single-vector retriever end to end.
DistilVDR distills bilaterally from a frozen 8B teacher (Qwen3-VL-Embedding-8B) into a 524M single-vector retriever: a 454M document encoder and a 70M query encoder.
The asymmetric design is the point. VDR inputs are inherently asymmetric: the query is text, the document is an image. The student mirrors that: the document side uses InternViT-300M plus ModernBERT-base to load up visual capacity, while the query side is just DistilBERT-base with a linear projection at 70M. The two students never share a forward pass during training, so the document-side and query-side distillations are fully decoupled.
The objective is minimal: a pointwise cosine alignment that pulls each student's output vector toward the teacher's frozen embedding-space target. Because the teacher was itself trained with relevance supervision, the student inherits retrieval ability through that space with no relevance labels, no negative sampling, and no contrastive term. Two variants, HiRes and Fast, share encoders and training and differ only in the document-side visual-tile budget: six tiles for HiRes, two for Fast.
Average NDCG@5 across the 22 datasets of ViDoRe v1+v2+v3:
| Model | Params | Type | v1 | v2 | v3 | Avg |
| Qwen3-VL-8B (teacher) | 8.1B | single-vec | 87.31 | 69.76 | 56.07 | 71.05 |
| DistilVDR-HiRes | 524M | single-vec | 82.81 | 55.34 | 47.07 | 61.74 |
| DistilVDR-Fast | 524M | single-vec | 81.34 | 54.95 | 43.66 | 59.98 |
| colSmol-500M | 478M | multi-vec | 82.42 | 43.09 | 33.52 | 53.01 |
HiRes retains 86.9% of the teacher. It leads the strongest sub-1B baseline, colSmol-500M, by 8.73 points and even edges past the 2.2B DSE-Qwen2 and 2.9B ColPali v1.3 on average. The advantage is largest on v3, the high-resolution-sensitive split (47.07 vs colSmol's 33.52). Decomposing the 9.31-point gap to the teacher, the document side accounts for 6.03 and the query side 4.69, which confirms the document side is the bottleneck and justifies stacking visual capacity there.
On efficiency, Fast reaches 99.04 docs/sec at 2.10 GB VRAM. The index is 16.4 GB per million documents, 15.6x smaller than the 256 GB of multi-vector baselines. Scoring 10K documents takes 9.6 ms versus 1100 to 3200 ms for multi-vector setups.
VDR is the entry capability for RAG over scanned files, PDFs, and screenshots, content where the text lives inside an image, and the cost wall for deployment is exactly indexing and serving. DistilVDR compresses an 8B model into a 524M single-vector retriever, shrinks the index by more than an order of magnitude, speeds up scoring by two orders of magnitude, and still leads the sub-1B field and beats some 2-3B models on average. For teams building large-scale document-retrieval indices over enterprise corpora, this is a directly usable engineering recipe, and the code is open source.
The authors are candid. The student is bounded by the teacher's embedding space and inherits its weaknesses. Baselines are compared as official releases rather than retrained under a matched budget, and there is no same-architecture contrastive-training control. Only one teacher family and scale is tested. The model still trails 4-8B multi-vector models by 7-10 points, especially on v3. The query tower is text-only and largely limited to Latin scripts; multilingual and non-text queries are not covered. Scalar quantization, product quantization, and binary hashing are untested. Whether quality transfers to enterprise corpora with in-house layouts is not validated.