Gemini 3.7 Flash Tops Zapier's AutomationBench, Beating Pricier Claude and GPT Models
_philschmid · x · 2026-08-14
Zapier's AutomationBench evaluates LLMs on end-to-end workflow execution using 47 real tools across six business functions: Sales, Marketing, Operations, Support, Finance, and HR.
Gemini 3.7 Flash ranks 1st with a 30.44% success rate and a cost of $0.61 per task, outperforming more expensive models like Claude Opus 5 and GPT-5.6 Terra. It dominates in Marketing, Finance, Sales, and Support, but trails behind Claude Opus 5 in Operations.
More from coding & agent
- Dispatch launches newsletter infrastructure built for AI agents, one API call to publish — RichardsonDx · 2026-10-16
- Creator shares the agent-powered system behind millions of monthly impressions — VibeMarketer_ · 2026-10-16
- Devs say LLM coding mistakes are a different class: overcomplication, inferred limits, worse at higher reasoning — Mu5_ · 2026-10-16
- How do you debug bad agent runs when you already have tracing? — Sensitive-Parsnip-12 · 2026-10-16
- Where does pre-task context live for agents? Devs debate workflows beyond handoff — shishen2026 · 2026-10-16
- No-instruction autonomous agent wakes up, sets own goals, and tries hacking by default — sethlazar · 2026-10-16