OpenAI web search can waste 87% of injected tokens, local pipeline matches 96% accuracy
Remote-Breadfruit204 · reddit · 2026-07-24
The author argues that OpenAI’s web search tool can be surprisingly expensive for agent loops because it injects a large amount of context back into the model.
They built a local, free alternative pipeline with:
- multi-engine search plus RRF fusion
- local page fetching
- hybrid BM25 + embedding retrieval
- cross-encoder reranking
- sentence-level compression
- exact and semantic caching
On a subset of OpenAI’s SimpleQA benchmark, they report:
- 96% accuracy, matching hosted search
- 87% fewer injected tokens
- sentence-level compression roughly halved result tokens without recall loss
- semantic caching makes paraphrased repeat queries free
- a 16-search test loop avoided about $1.49 in cost
The code and eval report are available in the linked GitHub repo.
Related event: Open-source Webfetch slashes AI agent search costs and tokens(4 posts)→
More from coding & agent
- DeepMind paper: LLMs can't sample randomly on their own, only convert given seeds — rohanpaul_ai · 2026-10-27
- DeepMind paper: LLMs can't sample randomly on their own — pass random numbers from code instead — rohanpaul_ai · 2026-10-27
- BELL: Open-Source Stack Combines Browserbase, Eve, iMessage and Stripe Link for Personal Agents — jeff_weinstein · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27