DECIDE benchmark tests voice-agent decisions on 4,867 questions from 1,081 real calls, scoring 20 models
Aiden_Tech_Ai · x · 2026-11-08
DECIDE is a new benchmark, built with Bolna, that tests what voice agents actually decide rather than transcribe: speak or wait, route the call, pick a tool, switch languages — signals like caller annoyance and interruptions that text alone can't capture.
- 4,867 decision questions drawn from 1,081 real production agent-caller clips (3–25 seconds each)
- 32 tasks across 6 families: turn-taking, graph routing, tool decisions, language switching, caller emotion, business intent
- Direct-audio models and STT+text pipelines (20 models total) scored on the same questions, isolating what audio actually adds
The authors stress that all evaluation uses real production calls, not scripted demos — the key differentiator from existing speech benchmarks.
More from coding & agent
- Built an interactive 3D optics course for GCSE physics entirely with Claude Opus 5.5 — techartist_ · 2026-11-08
- Perplexity CEO uses an agent to unmask anonymous recruiter candidates and reach them directly — AravSrinivas · 2026-11-08
- Alibaba's Qoder Launches Stealth Model Qoduck, Free for Individual Users — Aiden_Tech_Ai · 2026-11-08
- Viggle-Remake hands-on: wiring its MCP to ComfyUI workers with Claude for full video remake — thefi3nd · 2026-11-08
- Rebuilding a portfolio with a small team of Claude Code agents and handover files — WillowFar6967 · 2026-11-08
- Prompt Card: a local tool that extracts prompts and settings from AI-generated files — lordstryfe · 2026-11-08