Perplexity explains its embedding search: batch vs online workloads in one vector space
perplexity_ai · x · 2026-09-05
Perplexity explains that queries and documents are embedded into one vector space and searched by nearest vectors, creating two distinct workloads: throughput-focused bulk batch embedding for indexing and scoring, and latency-focused per-query online embedding for live search—each requiring different infrastructure optimizations.
Related event: Perplexity Unveils Its Self-Built SoTA Embedding Inference Stack(8 posts)→
More from Infra
- Sizing a 1 GW datacenter: ~522,000 Blackwell GPUs fit inside — bookwormengr · 2026-10-21
- DeepSeek's production density reportedly hits ~$38 per GPU-hour at saturation — bookwormengr · 2026-10-21
- PyTorch details KTIR and Dataflow Scheduler behind IBM's Spyre accelerator backend — PyTorch · 2026-10-21
- Indie dev replaces $1k OpenAI bill with a YAML pipeline on spot GPUs and vLLM — cmm324 · 2026-10-21
- DeepSeek open-sources DeepInfer, the vLLM-based engine behind V4.1-Flash — The Decoder · 2026-10-21
- Robotics startup launches Deployments to serve fine-tuned HF checkpoints to robots in real time — arnie_hacker · 2026-10-21