Artificial Analysis Benchmarks 20 Search APIs; Octen Costs ~$0.058 per 1,000-Task Task
ArtificialAnlys · x · 2026-09-11
Artificial Analysis launched its Search Index benchmark comparing 20 Search API products from 10 providers across DeepSearchQA (F1), BrowseComp (exact-match accuracy), and its own AA-Omniscience benchmark, with an identical model, harness, and settings isolating search quality as the only variable.
- Cost breakdown (Octen): roughly $9 per 1,000 tasks in search costs (6–12 searches per task at a discounted $1/1k queries) plus $49 per 1,000 tasks in model costs, since Octen's highlight payloads run 7k tokens per search.
- At $0.058 per task with an index size of 77, Octen sits on the cost/quality Pareto frontier.
- Artificial Analysis's own Search Index ranks top across all three benchmarks.
The benchmark targets developers building deep research, coding, and knowledge agents who need to weigh retrieval quality, result formatting, and the cost/speed tradeoff.
Related event: Octen Search Ranks Third in Benchmark with Fastest Speed, Lowest Cost(3 posts)→
More from Models
- Omni-InteractScope benchmarks proactive interaction in continuous omnimodal streams — Qwen · 2026-10-28
- Mistral releases Timestral time-series foundation models, tops GIFT-Eval — mistralai · 2026-10-28
- Gemini 4 Argon Hits 1.73% WER on OCR Benchmark, Surpassing Human Experts — gabriberton · 2026-10-28
- Meta's Omnilingual MT covers 1,600 languages, 1B–8B models rival 70B LLMs — kalyan_kpl · 2026-10-28
- Uncensored LFM3-1.7B GGUF weights released on Hugging Face — kalyan_kpl · 2026-10-28
- Token demand on OpenRouter grew 27x to 165T weekly, open models now serve 72% — FinanceYF5 · 2026-10-28