OpenAI web search can waste 87% of injected tokens, local pipeline matches 96% accuracy
Remote-Breadfruit204 · reddit · 2026-07-24
The author argues that OpenAI’s web search tool can be surprisingly expensive for agent loops because it injects a large amount of context back into the model.
They built a local, free alternative pipeline with:
- multi-engine search plus RRF fusion
- local page fetching
- hybrid BM25 + embedding retrieval
- cross-encoder reranking
- sentence-level compression
- exact and semantic caching
On a subset of OpenAI’s SimpleQA benchmark, they report:
- 96% accuracy, matching hosted search
- 87% fewer injected tokens
- sentence-level compression roughly halved result tokens without recall loss
- semantic caching makes paraphrased repeat queries free
- a 16-search test loop avoided about $1.49 in cost
The code and eval report are available in the linked GitHub repo.
Related event: Open-source Webfetch slashes AI agent search costs and tokens(4 posts)→
More from coding & agent
- Anthropic researcher: 99% of engineers now run swarms of 300+ self-improving agents — AlishaOutridge · 2026-09-11
- Gergely Orosz: Shipping 10x PRs With AI Agents, Sites Fill With Small Regressions — ducha_aiki · 2026-09-11
- Same Echo Maze prompt, three frontier models: all passed visually but shipped the same hidden bug — eyishazyer · 2026-09-11
- Astra storyboards plus Minimax H3 per-shot generation boost video success rates — Hailuo_AI · 2026-09-11
- Codex tip: use Sol with Astra and Luna sub-agents to save usage — pvncher · 2026-09-11
- agents-best-practices: a provider-neutral Agent Skill for designing and auditing agentic harnesses — tom_doerr · 2026-09-11