Visa's agentic commerce benchmark: 57.2% pass rate, agents buy in 54% of failures
krishnan · x · 2026-11-10
Visa's Agentic Commerce Benchmark (released Nov 5) ran 22 models from 12 labs through 15 shopping scenarios, testing whether agents correctly decline to buy when nothing fits the customer's requirements.
Key numbers: overall pass rate 57.2%; only 34.2% of scenarios passed on all three attempts; and 54% of failed runs still ended in a purchase.
Caveats: these are benchmark buy/don't-buy decisions, not real spending, and Visa has a stake in agentic commerce taking off. Conversion isn't customer success — an agent rewarded on checkout rate will tend to close.
Practical advice: write down customer requirements before any pilot run, then measure both failure modes — unsuitable purchases the agent should have stopped, and unnecessary refusals when a good fit existed.
More from coding & agent
- LlamaParse's five deployment options span SaaS to your own GPUs — llama_index · 2026-11-10
- Building a prediction-market agent: when should it trust its 34.5% over the market's 20%? — Dry-Comfortable-2678 · 2026-11-10
- NVIDIA Paper AgentBaton Tackles Efficient Multi-Model Collaboration in Agentic Tasks — _akhaliq · 2026-11-10
- OpenClaw ships v2026.11.2 patch: 113 PRs fix broken updates, missing reminders, duplicate widgets — dean_rie · 2026-11-10
- Hooking a cloud AI agent to your home network: Muse + Tailscale + MCP drives RC cars via SDR — Dave_Maynor · 2026-11-10
- Claude Duet Connects to Claude Code on Your Mac: Code, Edit Video From Your Phone — dean_rie · 2026-11-10