Halve Your Multi-Vector Index With HierarchicalTokenPooling at 100.6% BEIR
tomaarsen · x · 2026-08-18
Late interaction is the state of the art for visual document retrieval: text queries against page images with charts and tables intact, no OCR — ColPali-family checkpoints run through the exact same two calls, with MaxSim scoring query text tokens against image patches. The post also covers HierarchicalTokenPooling (Clavié, Chaffin & Adams), which clusters each document's token vectors with Ward linkage and keeps 1/poolfactor of them: poolfactor=2 halves the index at 100.6% of unpooled BEIR performance. It can be applied per call, standalone, or baked into the model.
More from coding & agent
- Agents turn conservative as context grows: edits shrink from 150 to 30 lines — dbreunig · 2026-10-16
- ai& becomes an official built-in Cline provider with 13 open-weight models — DavidBennett__ · 2026-10-16
- Open-source rizin-mcp lets LLMs run local binary and malware analysis via Rizin — Pure_Emergency1424 · 2026-10-16
- x402 as a free GTM channel: agents discover 100+ tools, no API keys needed — 0xJeff · 2026-10-16
- Sam Altman: stop getting better at prompts, build loops and graphs that prompt for you — goyalshaliniuk · 2026-10-16
- POV ski game on real Matterhorn elevation data, built with Claude Opus 5.5 — techartist_ · 2026-10-16