AA Benchmark: Parallel and Firecrawl lead the Pareto frontier for search cost and performance
ArtificialAnlys · x · 2026-08-19
Artificial Analysis benchmarks show that Parallel (turbo/advanced) and Firecrawl form the Pareto frontier for the Search Index vs. Cost per Task, representing the best blended performance available today.
Key findings:
- Firecrawl Search: Scores 73 on the AA-Search Index with a total cost of $0.075 per task (40% is search cost). However, it has the slowest time per task (55.0s).
- Parallel (advanced): Costs 12% more per task to reach an Index of 75. It reduces model inference costs due to superior search efficiency.
Methodology: Models run inside the open-source Stirrup agent harness with web search/fetch tools, limited to 25 turns per task.
Related event: Artificial Analysis Launches Search API Benchmark, Parallel and Exa Lead(3 posts)→
More from Apps
- AI circuit builder that runs real simulations wins $12k at Replit Designathon — amasad · 2026-08-19
- Linear CEO: Design will be AI's biggest problem — every · 2026-08-19
- Developer visualizes Stripe dashboard as a growing forest — round · 2026-08-19
- Computer use brings the age of real consumer agents — venturetwins · 2026-08-19
- Reflection: The Model Didn't Get Worse, My Prompts Got Lazier — ClickOk5811 · 2026-08-19
- OpenAI Marketing Team Demo: Drafting Launch Blogs with ChatGPT Work — OpenAI · 2026-08-19