Paper: LLMs Beat Embeddings Slightly but Cost 1,431x More
Muennighoff · x · 2026-08-20
A new paper titled "The Embedder's Dilemma" compares 10 LLMs against 26 embedding models across 37 tasks. Results show LLMs slightly lead overall (77.6 vs 77.2), particularly excelling in reasoning-heavy retrieval. However, the cost is massive: achieving comparable quality with an LLM can cost up to 1,431x more than an embedding model ($154 vs $0.11), with inference speeds 2.5 to 736x slower. The study advises a division of labor: use embeddings for similarity, classification, and clustering, and reserve LLMs for reasoning-intensive retrieval.
Related event: Study: LLMs Marginally Beat Embedding Models but at 1000x Cost(2 posts)→
More from Infra
- UK AI Chip Startup CallosumAI Raises $100M Seed — HZoete · 2026-08-20
- UK Chip Startups Raise Over $900M in Two Weeks — HZoete · 2026-08-20
- Loudoun County: Most data centers and highest median income in US — kevinnbass · 2026-08-20
- Rust Compiler Contributor Sets Goal: Make rustc 10x Faster — mitsuhiko · 2026-08-20
- Why hasn't AI pricing increased to its 'true cost' yet? — MacIntoic · 2026-08-20
- Obscura: Rust-based headless browser for AI agents — tom_doerr · 2026-08-20