Artificial Analysis Benchmarks 20 Search APIs; Octen Costs ~$0.058 per 1,000-Task Task
ArtificialAnlys · x · 2026-09-11
Artificial Analysis launched its Search Index benchmark comparing 20 Search API products from 10 providers across DeepSearchQA (F1), BrowseComp (exact-match accuracy), and its own AA-Omniscience benchmark, with an identical model, harness, and settings isolating search quality as the only variable.
- Cost breakdown (Octen): roughly $9 per 1,000 tasks in search costs (6–12 searches per task at a discounted $1/1k queries) plus $49 per 1,000 tasks in model costs, since Octen's highlight payloads run 7k tokens per search.
- At $0.058 per task with an index size of 77, Octen sits on the cost/quality Pareto frontier.
- Artificial Analysis's own Search Index ranks top across all three benchmarks.
The benchmark targets developers building deep research, coding, and knowledge agents who need to weigh retrieval quality, result formatting, and the cost/speed tradeoff.
Related event: Octen Search ranks third in Artificial Analysis search benchmark(2 posts)→
More from Models
- Reddit: No Major LLM Has Ever Been More Sensitive Than Today's Claude — PressPlayPlease7 · 2026-09-11
- mythos still reasons like it's in a CTF, nearly burns its IP and considers contacting a human for the first time — voooooogel · 2026-09-11
- Genspark launches Gen-1 Slides, a work model priced at 1/17th of Opus 5 — Scobleizer · 2026-09-11
- OpenAI's Rumored Internal Model 'Bel' Could Be the Most Hyped Release Ever — imadade · 2026-09-11
- Model infers OSS software behavior from source snippets memorized in pretraining — voooooogel · 2026-09-11
- DeepSeek-V4.1-Flash rolls out to Ollama cloud Pro subscribers after Max and Team debut — ollama · 2026-09-11