Study: LLMs Cost 1,431X More Than Embeddings for Retrieval
rohanpaul_ai · x · 2026-08-22
A joint Harvard and Stanford paper suggests keeping embedding models for cheap, fast candidate retrieval instead of replacing them with LLMs. Across 37 tasks, Gemini 3.1 Pro and Octen-8B achieved a statistical tie (77.6 vs 77.2), but the LLM cost 1,431x more ($154.14 vs $0.11). The study recommends using embeddings for retrieval and only bringing in LLMs when the shortlist requires reasoning.
Related event: Harvard-Stanford Paper: LLMs Match Embedding Models but Cost ~1431x More(6 posts)→
More from Research
- Vals Science Index: OpenAI Models Lead Overall, but No Model Sweeps Every Science Discipline — BenBlaiszik · 2026-10-17
- First-author PhD paper frames AI text detection as continual learning, accepted at NeurIPS 2026 — lucy3_li · 2026-10-17
- Regents Labs curates 17 AI/agent science papers weekly with AI-generated deep reads — seanwbren · 2026-10-17
- A 'pre-numeric' AI math layer trains patterns before numbers exist — debreuil · 2026-10-17
- research-rag: local CPU-only PDF/ebook search with browser, CLI and MCP access — kishkisan · 2026-10-17
- Epoch AI's SpireClimb: models beat human author at Slay the Spire but still err each act — Jsevillamol · 2026-10-17