Open-source PlaceBench tests 6 voice assistants on 150 real-place questions: Dots 92, Ask Maps 49
whowasntright · reddit · 2026-11-21
A Reddit writeup of PlaceBench, an open-source (CC-licensed) benchmark of 150 real-world venue questions across 121 cities, grading six AI assistants against each venue's own website, notices, and transit timetables. Wrong answers subtract a full point.
- Dots: 92/100, zero wrong answers, 57s median
- Muse 83 (6 wrong), Claude Duet 79 (6), Instinct 76, Grok Bot 73 (12)
- Ask Maps ranked last at 49/100 with 30 wrong answers out of 150, despite being fastest (8s median) — fast guessing hurts
Two failure modes stand out: 12 answers pointed people to closed or relocated venues, and repeating 20 questions three times caused answer flips (Instinct was consistent on only 5/20). The author argues map snapshots lag venue counters on live edge questions like tonight's reservations or holiday hours, hence pairing lookup with phone verification — and promotes PlaceCall, an AI calling service for US businesses that discloses it's an AI on a recorded line and bills $0.10 per quoted answer, $0 on holds.
More from coding & agent
- AWS Marketplace rolls out AI agents that can autonomously buy software for companies — jeremyakahn · 2026-11-21
- CT-Bench, a benchmark for LLM agent cooperation via self-negotiated contracts, accepted at COLM 2026 — maxhkw · 2026-11-21
- Google's Pro² Agent hits 76% medal rate on MLE-Bench, a new record — dair_ai · 2026-11-21
- App plus new database cluster deployed in 12 seconds with Cloud Run --no-build and AlloyDB Express — rseroter · 2026-11-21
- Claude Code 2.1.330 ships with 142 CLI changes and confirm-before-risky-actions policy — ClaudeCodeLog · 2026-11-21
- Claude builds Miami-style town with 2,538 Blender assets, rendered in Unreal in 74 minutes — piotrbinkowski · 2026-11-21