Web search grounding cut a calling agent's booking accuracy by 5+ points across 2,415 tests
Evening_Hawk_7470 · reddit · 2026-11-19
Based on the Local Search Benchmark from voygr and placecall (345 prompts, 50+ cities, 2,415 evaluations of gpt-5.2, gemini-2.5-flash, claude-sonnet-4-5, and sonar-pro, verified against ground truth and real phone calls), the author shares three counterintuitive findings:
- Search helps discovery, hurts execution: grounding boosted general discovery by +10 to +21 points (gpt-5.2 led at 90.7 overall), but claude-sonnet-4-5 and gemini-2.5-flash each lost over 5 points on transactional tasks because search engines rank Yelp/TripAdvisor pages and old blogs with stale forwarding numbers above the venue's real line.
- Search never erased closed businesses: without search, 20% of claude-sonnet-4-5 results had fatal errors; search cut that to 8% at best, yet all 7 configurations confidently recommended booking a Buenos Aires restaurant closed for over a year.
- Constraint accuracy varied 16 points (85% down to 69%) since directories are static snapshots missing live capacity.
Architectural lesson: keep rough search separate from pre-dial phone validation. For US use, pair place lookup with PlaceCall ($0.10 per valid staff answer, free on dead lines); Google's "Let Google call" suits single personal checks; Vapi/Retell work for inbound or non-US stacks if you scrub aggregator numbers yourself.
More from coding & agent
- Higgsfield Launches MCP 2.0 for Claude, Bringing Its Creative Studio Into Chat — piotrbinkowski · 2026-11-19
- CoRL workshop: running Karpathy's autoresearch on real robots — the sandbox, not the model, is the bottleneck — Kyrannio · 2026-11-19
- OpenAI data: 90th-percentile researcher now spends $23,000 on agentic AI tools — kliu128 · 2026-11-19
- Gemini 4 Neon lands Day 0 in LangChain's Managed Deep Agents for long-running agent work — hwchase17 · 2026-11-19
- badlogic: agents 'reward-hack' past your lint rules and tests—without insight under the hood, your verifications may be worthless — teropa · 2026-11-19
- Color skill flips GPT-6.2 Astra's TiKZ unicorn art from cartoon to flat vector style — Angaisb_ · 2026-11-19