Testing 13 Agent Search APIs: Hidden Token Costs Vary by 67x
Patient-Injury-1327 · reddit · 2026-08-07
The author tested 13 popular search APIs and revealed that the biggest hidden cost in agent search isn't the per-request fee, but the LLM token cost required to read the returned payload.
- Methodology: Ran the same 100 questions across all APIs using a single search call and the same reader model, tracking actual billed token counts.
- Cost Discrepancy: Due to varying payload sizes, the total cost difference between the cheapest (serpdive-krill at $2.91/1k) and the most expensive (exa-search at $196.63/2k) is a massive 67x.
- Accuracy: With a sample size of n=100, accuracy carries a ±10% margin of error. you-web (95%), firecrawl-search (94%), and brave-search (93%) scored the highest.
- Dynamic Answers: 13 out of 100 answers had changed over time. Engines returning updated facts were falsely penalized by the stale answer key. All payloads and code are open-sourced for reproducibility.
More from coding & agent
- Axelrod Labs (YC S26): Deploying AI Agent Workforces to Automate Hotel Operations — ycombinator · 2026-08-07
- Kokoro Web: Open-Source Browser-Based TTS with OpenAI API Compatibility — tom_doerr · 2026-08-07
- vLLM Runs on 500,000 GPUs: a16z Talks with Core Maintainer Simon Mo — a16z · 2026-08-07
- Sentry Open-Sources dotagents: Unify Coding Agent Configs with One TOML File — zeeg · 2026-08-07
- Killing Figma: Building Websites with Claude Code and AI Media Stack — MicahBerkley · 2026-08-07
- Open-Source Sky-Agent: 1-Click Cloud Deployment for Any Agent at $7/Month — Rude_Substance_8904 · 2026-08-07