Shopify's RoutingTrie lifts generative retrieval Hit@100 from 67.5% to 96.1% with 1.1ms overhead
_reachsumit · x · 2026-11-11
Shopify researchers publish RoutingTrie on arXiv: in generative retrieval, metadata constraints are enforced before beam selection rather than filtered after generation.
- One shared semantic-ID trie, with a compact Dense Bitset or Sparse List index per metadata field, guided by a storage-cost formula
- During decoding, field probes produce a combined mask so beams only expand along edges satisfying requested constraints
- In a production replay, Hit@100 rises from 67.5% to 96.1% (+28.6pp) with only 1.1ms added latency
- ShopTrie benchmark generator, scripts, and CPU/Triton GPU implementations will be released
Highly relevant for teams building catalog-style generative retrieval.
More from Infra
- Anthropic says Claude ported kernels to run on AMD GPUs in just 3 days — scottleibrand · 2026-11-11
- Chick-fil-A runs 2,800 edge Kubernetes clusters with billions of monthly MQTT messages — SumitGup · 2026-11-11
- RX 6800 gets 3x faster ComfyUI images by re-enabling silently disabled INT8 — TheCat001 · 2026-11-11
- Hybrid product search straight from the data lake: Pinot + BM25 + RRF hits 6.5-9ms p50 on 277M products — _reachsumit · 2026-11-11
- CoreWeave revenue jumps 168% to $3.66B but net loss balloons 6x to $709M on borrowing costs — rohanpaul_ai · 2026-11-11
- NVIDIA open-sources NVKE and NVKR for managed Kubernetes on GPU infrastructure — HowDevelop · 2026-11-11