TEngineDB-V: An OLAP-Native Vector Search System for Large-$k$ Workloads at Tencent
Xufei Wu, Pengcheng Zhang, Yitong Song, Xiaobo Zhang, Anqi Liang, Kai Wang, Jijun Du, Yidi Xiong, Guangxu Cheng, Zhe Chen, Peng Chen, Guoliang Li, Xuanhe Zhou, Fan Wu
cs.DB
2026-08-01
TEngineDB-V turns large-k vector search into OLAP-native relational operators. On a 10-billion-image cluster it returns top-100k results in under 5s, up to 145x faster than StarRocks.
Vector search is no longer just "find the 10 most similar rows." At Tencent, text-to-image teams pull 10k-100k images at a time from a 10-billion-image corpus to assemble fine-tuning sets, and advertising teams圈 a large similar set and then aggregate over it. This large-k retrieval (k from 10^3 to 10^5) tolerates more latency than RAG but is followed by filtering, aggregation, and joins, so it is an analytical workload.
Existing systems fail on both sides. Specialized vector DBs (Milvus, PGVector, ElasticSearch) cap k at 10k-16k to control tail latency and offer weak analytics. OLAP engines (StarRocks, Doris) bolt a vector index on as a black box, building one local index per data segment and running scatter-gather: broadcast the query to all segments, compute a local top-k in each, then merge globally. With N segments each returning k, the result volume inflates to N times k (200M rows for k=100k and N=2000), and disk I/O, network, and merge cost explode.
TEngineDB-V makes vector search a first-class operator in the OLAP engine, restructured across three layers.
The storage layer uses a segment-decoupled global index. Instead of a local index per segment, Spark builds one global IVFPQ index over all data and materializes it as three relational tables: an IVF centroid table (sharded by cluster ID), a PQ codebook table (unsharded), and a quantized-vector table (sharded by cluster ID). A query probes only relevant clusters, so cost scales with the number of probed clusters, eliminating scatter-gather. The index refreshes asynchronously with an atomic swap and accepts eventual consistency (deleted IDs may linger, new inserts wait for the next rebuild), which fits analytical workloads that update every few days.
The compute layer decomposes IVFPQ search into relational operators: IVF Prune (scan the centroid table for the nearest N clusters), LUT Compute (scan the codebook to build a distance lookup table), and Distance Estimate (broadcast the LUT to candidates, accumulate, then TopK). Once decomposed, standard OLAP tricks apply: late materialization, join runtime filters, columnar FastScan, operator fusion. They add DPPQ: conventional PQ minimizes Euclidean reconstruction error, but nearest-neighbor ranking is more sensitive to direction, so DPPQ minimizes angular deviation and refines residuals hierarchically over epochs. At equal bit budget it beats both PQ and RaBitQ on recall, and crucially folds refinement into the operator, avoiding the raw-vector re-ranking stage (traditionally 2k-3k candidates, which dominates latency at large k).
The control layer feeds index semantics to a Cascades-based optimizer. For filtered FANNS queries it enumerates eight plans (pre- vs post-filter, build/probe side, broadcast vs shuffle join) and picks the best with a cost model that jointly models CPU, memory, and network, weighted 0.5, 2.0, 1.5.
| Setting | Metric | TEngineDB-V | Comparison |
| Wikipedia, k=20k, recall 0.86 | latency | 332ms | 2.7/8.9/7.7/50x faster than Milvus/StarRocks/PGVector/DiskANN |
| SIFT1B, recall 0.88 | latency | 208ms | 137/13/105x faster than StarRocks/PGVector/DiskANN (Milvus OOM) |
| SIFT1B, k=10k | latency | 189ms | 145.5x faster than StarRocks |
| Production Tencent-Image, 10B, recall 0.88 | latency | 14.2s | legacy 200s, StarRocks 333s, Milvus 56s |
| Production Tencent-Image, recall 0.8 | vs legacy | 52x faster | 65x vs StarRocks, 26x vs Milvus |
DPPQ at 400 bits/vector on SIFT1M beats PQ by 4.1 points of recall and RaBitQ by 3.2. Columnar FastScan runs only about 5% slower than row-oriented FastScan and 4-5x faster than no FastScan. The largest production cluster is 30+ nodes, 100TB, about 10B image embeddings, with top-100k search stable under 5s.
Large-k vector search is becoming a real workload: LLM data curation, multimodal analytics, ad selection all pull a big batch and then process it. The takeaway is that you do not have to bolt on a separate vector DB. Folding vector search into the OLAP engine as relational operators lets it be jointly optimized with joins, filters, and runtime filters, and scales to 10 billion. For any team building an analytical vector pipeline, this is a directly reusable industrial design. One caveat: it is built for low-update, eventually consistent analytics, not a real-time transactional store.
Eventual consistency means the index lags the base table within a refresh window: deleted IDs may still be returned and new inserts wait for the next rebuild, unsuitable for latency-sensitive freshness.
Recall is only evaluated at 0.8-0.9 because in their pipeline vector search is coarse candidate generation, not the final answer; under high-recall requirements the advantage may shrink.
The three cost-model weights (0.5/2.0/1.5) are hand-tuned to their hardware and may not transfer directly.
DPPQ stores an extra FP32 scaling factor per subspace per refinement epoch, so storage grows with epoch count; the paper does not give a full storage accounting.