6 Search APIs Tested on 2,182 Questions: cloro Leads Multi-hop at 80.2%
rbatista191 · reddit · 2026-11-12
The cloro team benchmarked six web search APIs (cloro, SerpApi, Parallel, Exa, Brave, Tavily) on 2,182 questions with one search and 10 results each, spanning SimpleQA Verified, FreshQA, and FRAMES for 13,092 graded answers.
Key results
- Short factual questions: Parallel 97.8%, Exa 97.3%, cloro 96.9% — essentially tied
- Recent events: SerpApi 88.5% and cloro 88.3% led
- Multi-hop opened a gap: cloro 80.2% > SerpApi 77.4% > Parallel 65.3% > Exa 64.0% > Brave 53.9% > Tavily 48.4%
- Speed runs the other way: Exa's 0.74s median is fastest; cloro slowest at 10.96s
Most useful finding: keeping Google results fixed and changing only what the model reads — removing the AI Overview dropped accuracy from 99% to 86%; fetching full page text added little while costing 8x tokens, and only 56% of pages returned any text (22% blocked by robots.txt).
Practical takeaways: for single-search multi-hop, use a Google SERP API that surfaces the AI Overview; pick Exa when latency matters; for simple lookups choose on price and speed; don't fetch full pages by default. The author notes this measures search APIs, not a browsing agent, and asks whether the multi-hop gap survives a full agent loop.
More from coding & agent
- Vibe-coding with Claude: reviewing output quality is what separates slop from good work — eptwts · 2026-11-12
- His ChatGPT Pro agent swarm burned 30% of quota overnight teaching itself to draw — amplifiedamp · 2026-11-12
- Hornet launches with thesis: retrieval infrastructure is mandatory for the agent era — jobergum · 2026-11-12
- Six rules to stop coding agents from falsely reporting tasks as done — Al_Grigor · 2026-11-12
- What separates a demo from a production AI agent: a 5-point eval checklist — ghumare64 · 2026-11-12
- Broke student's vibe coding stack: ChatGPT writes the plan.md, free agents do the work — lo_siento0 · 2026-11-12