Octen Search debuts third on search index at 16.9s per task, $0.058 cost
ArtificialAnlys · x · 2026-09-11
Artificial Analysis benchmarked Octen Search using a fixed base model in its open-source agent harness Stirrup, changing only the search provider:
- Third on the Search Index (77), behind Perplexity Search medium (80) and high (79), tied with Perplexity low; scored 86 on BrowseComp, one point off the lead, and 80 on DeepSearchQA, though its AA-Omniscience (66) trails Firecrawl (73)
- Fastest provider: 0.2s per query, 5x faster than Perplexity variants; 16.9s per task, the lowest of any result and below the 22.6s model-only baseline
- Cheap: $1 per 1k queries (80% off standard $5/1k), 6–12 searches per task ($9/1k tasks) plus $49/1k model cost; at $0.058 per task it sits on the Pareto frontier between TinyFish ($0.035, index 71) and Perplexity medium ($0.091, index 80)
Related event: Octen Search ranks third in Artificial Analysis search benchmark(2 posts)→
More from coding & agent
- Ollama lets you customize ChatGPT's model selector with local and cloud models — ollama · 2026-09-11
- ChatGPT Desktop Codex app can now use local models via Ollama 0.34 — ollama · 2026-09-11
- Inspect evals: default continuation prompt can make agents misread bad actions as approved — AdtRaghunathan · 2026-09-11
- Claude Code creator: AI-written production code should be held to a higher bar — bcherny · 2026-09-11
- Dev culture gap: some burn a week of Codex quota daily, others never tried it — yihui_indie · 2026-09-11
- Modern Context Stack: a satire site skewering the data industry's tool-buying habit — juansequeda · 2026-09-11