hpool Enables Flexible and Adaptive Compression for Retrieval Models

antoine_chaffin details the technical features and evaluations of hpool, a retrieval model compression method. This approach breaks the limitations of traditional fixed compression, offering a versatile trade-off between performance and compression, and proves competitive against recent baselines.

Key Details and Flexibility

hpool allows users to set the compression degree flexibly at inference by cutting directly at any position in the dendrogram. Additionally, the author suggests a multi-budget training strategy (training across various budgets like [4, 8, 16, 32, 64, 128, 300]), making the model easier to pool at any compression level. However, if the deployment target is fixed (e.g., specifically for r=32), dedicated training can retain 1.6% more information under the same λ than multi-budget training.

Adaptive Document Compression

Since documents vary in information density and compressibility, hpool supports adaptive pooling for each document. By dynamically setting the budget per document, it calculates the inter-cluster distance threshold required to hit an average target budget, avoiding the rigid approach of compressing all documents to the exact same number of tokens.

Comparison and Controversies with AGC

A recent paper on Attention-Guided Clustering (AGC) indicated that attention-learned pooling achieves better information retention than hpool on unregularized models. However, antoine_chaffin points out that the conclusion shifts for regularized models: across 7 evaluated BEIR datasets, AGC's average retention rate is 97.4%, slightly lower than hpool's 97.9%. The author cautions that these results are not directly comparable due to differences in training data and scale.

2026-07-07 ~ 2026-07-07 · 5 related posts