Inference-free SPLADE: retrieval at BM25-like query cost without per-query inference
qdrant_engine · x · 2026-09-22
KShivendu is presenting Inference Free SPLADE models at Doug Turnbull's Retrieval Augmented Gathering.
- Standard SPLADE uses a transformer to expand document keywords so keyword search can match synonyms and semantically similar terms, but it runs a model on every query — about 50ms p50 latency on CPU, and you need a GPU to go faster.
- The inference-free variant moves model work upfront onto documents at indexing time, making query cost roughly the same as BM25.
- The talk covers how it works, the trade-offs, and how to take it to production.
More from Infra
- DeepSeek reportedly bets on Huawei chips to train next-gen models; Liang says it 'has to work' — kimmonismus · 2026-09-22
- Modded RTX 3080 20GB laptop GPUs appear on AliExpress for AI use — thatguyjames_uk · 2026-09-22
- Whittle distills on HF: 27B-A3B MoE quant claimed to run on 8GB VRAM laptops — depressedclassical · 2026-09-22
- Qdrant benchmark: post-upload latency spikes are optimizers, tuned configs yield up to 100x faster search — qdrant_engine · 2026-09-22
- What Cloudflare can't do: D1 caps at 10GB, is single-threaded, and no real Postgres — Paimaamu · 2026-09-22
- Rackspace joins NVIDIA Cloud Partner Program with Blackwell pods for regulated enterprises — DavidLinthicum · 2026-09-22