ColPali-style visual document indices can be inverted: 47% of words recovered, source page ranked first 98.4%
_reachsumit · x · 2026-10-08
New research shows multi-vector visual document retrievers like ColPali are vulnerable to index inversion. Since each page is stored as 1,000 patch vectors in raster order, an attacker can reproduce pages via conditional document image generation. On ViDoRe v3, inverted pages recover 47% of words and 45% of sensitive tokens and rank their source page first 98.4% of the time. Token pooling and shuffling cut word recall to 8%, but a model restoring shuffled order raises source-page ranking from 3.8% back to 93.5%. The same attack transfers to another retriever at 70.2%. Indices should be treated as page-sensitive data.
Related event: Multi-Vector Visual Document Indexes Can Be Inverted to Reveal Page Content(2 posts)→
More from Safety
- Devs debate session-level runtime authorization on top of MCP's static config for agents — Longjupping_Tax_3598 · 2026-10-08
- Cryptographer: AI doing amazing math puts elliptic curve hardness in no danger — markjeffrey · 2026-10-08
- The AI safety debate has a framing problem: nobody actually argues against safety — Smart_Fly_5783 · 2026-10-08
- AI could end encryption as we know it: math breakthroughs threaten public-key crypto — sebkrier · 2026-10-08
- Fields Medalist Terence Tao reposts call for mathematicians to cut ties with OpenAI — Eliv_nurotic · 2026-10-08
- Ex-OpenAI Policy Head Miles Brundage: Deep AI Policy Thinking Is Impossible Amid the Chaos — Miles_Brundage · 2026-10-08