Paper: LLMs Beat Embeddings Slightly but Cost 1,431x More
Muennighoff · x · 2026-08-20
A new paper titled "The Embedder's Dilemma" compares 10 LLMs against 26 embedding models across 37 tasks. Results show LLMs slightly lead overall (77.6 vs 77.2), particularly excelling in reasoning-heavy retrieval. However, the cost is massive: achieving comparable quality with an LLM can cost up to 1,431x more than an embedding model ($154 vs $0.11), with inference speeds 2.5 to 736x slower. The study advises a division of labor: use embeddings for similarity, classification, and clustering, and reserve LLMs for reasoning-intensive retrieval.
Related event: Harvard-Stanford Paper: LLMs Match Embedding Models but Cost ~1431x More(6 posts)→
More from Infra
- Agentic CPU Guidebook: Why Agent Workloads Point to a $221B Server CPU Market by 2030 — BenBajarin · 2026-10-16
- Haight Street Compute Market: a one-day trading floor for GPU deals in SF — ctjlewis · 2026-10-16
- Temporal Serverless Workers on Cloud Run: queue-driven autoscaling, scale to zero — rseroter · 2026-10-16
- LMA brings day-0 model support to TPU for the first time, spanning NVIDIA, AMD, Intel via SGLang — ying11231 · 2026-10-16
- Semtech's analyst-day slide cleanly explains the four signal-integrity impairments — BenBajarin · 2026-10-16
- Fortytwo Engine boosts Qwen3.8-27B accuracy 7% and cuts tokens 25% via Swarm Inference — rohanpaul_ai · 2026-10-16