OpenAI web search can waste 87% of injected tokens, local pipeline matches 96% accuracy
Remote-Breadfruit204 · reddit · 2026-07-24
The author argues that OpenAI’s web search tool can be surprisingly expensive for agent loops because it injects a large amount of context back into the model.
They built a local, free alternative pipeline with:
- multi-engine search plus RRF fusion
- local page fetching
- hybrid BM25 + embedding retrieval
- cross-encoder reranking
- sentence-level compression
- exact and semantic caching
On a subset of OpenAI’s SimpleQA benchmark, they report:
- 96% accuracy, matching hosted search
- 87% fewer injected tokens
- sentence-level compression roughly halved result tokens without recall loss
- semantic caching makes paraphrased repeat queries free
- a 16-search test loop avoided about $1.49 in cost
The code and eval report are available in the linked GitHub repo.
Related event: Open-source Webfetch slashes AI agent search costs and tokens(4 posts)→
More from coding & agent
- A plugin lets agents control Codex Micro lights for email, Stripe and subagents — dkundel · 2026-07-24
- LangChain shows how Rillet uses LangSmith to monitor AI agents across 500+ customers — LangChain · 2026-07-24
- Localbrain turns any app into an offline, OpenAI-compatible local AI service — Everglow915 · 2026-07-24
- Nous Research’s Hermes Agent sends its first message in Buzz — Teknium · 2026-07-24
- Open-source WordPress MCP plugin gives agents least-privilege access, not admin keys — wpninjapro · 2026-07-24
- Agent timeouts expose a missing model for retry, verification, and compensation — Technical_Bench_188 · 2026-07-24