AA Benchmark: Parallel and Firecrawl lead the Pareto frontier for search cost and performance
ArtificialAnlys · x · 2026-08-19
Artificial Analysis benchmarks show that Parallel (turbo/advanced) and Firecrawl form the Pareto frontier for the Search Index vs. Cost per Task, representing the best blended performance available today.
Key findings:
- Firecrawl Search: Scores 73 on the AA-Search Index with a total cost of $0.075 per task (40% is search cost). However, it has the slowest time per task (55.0s).
- Parallel (advanced): Costs 12% more per task to reach an Index of 75. It reduces model inference costs due to superior search efficiency.
Methodology: Models run inside the open-source Stirrup agent harness with web search/fetch tools, limited to 25 turns per task.
Related event: Artificial Analysis Launches Search Index Benchmark for Search APIs(5 posts)→
More from Apps
- AI social gaming platform Astrocade hits 10 million monthly active players — andrey_kurenkov · 2026-10-16
- No-dev-experience game built in 6 weeks with Claude orchestrating Unity and asset pipeline — Gambo7592 · 2026-10-16
- 75-80% of my AI requests now go through Dots: a power user's month-long review — nickbaumann_ · 2026-10-16
- Techpoint Digest: Nigeria's NCC blocks fake devices, WhatsApp adds account-free calls — Fowe · 2026-10-16
- Nous Research's Open-Source Hermes Agent Makes TIME's Best Inventions of 2026 — NousResearch · 2026-10-16
- Solo dev spent 3 years building an LLM-powered D&D-style RPG with OpenAI models — tschilpi · 2026-10-16