DIAL Benchmark Ranks Speech-to-Speech Models With 30k+ Human Ratings Across 9k Live Calls
sarahcat21 · x · 2026-10-29
The team at withdavidai released DIAL (Dialogue Interaction Assessment Leaderboard), billed as the most comprehensive human-preference benchmark for speech-to-speech models, built on 30k+ ratings from 9k+ live multi-turn calls.
DIAL's premise: a good voice conversation takes more than a good first response. It evaluates full interactions across conversational timing, context tracking, groundedness, task completion, and audio quality, spanning consumer conversations, business calls, and agents calling on a user's behalf.
The sharer notes S2S models will eventually replace cascaded pipelines for many use cases, but evaluating them has been hard — making this benchmark timely.
More from Models
- Anthropic: removing 'think carefully' prompts cut Opus 5.5 latency from 8.2s to 2.1s — FinanceYF5 · 2026-10-29
- Anthropic releases Claude Opus 5.5 prompting guide with 11 best practices — FinanceYF5 · 2026-10-29
- Particle-1 inference goes live on OpenRouter, claiming 28-52% fewer output tokens — ycombinator · 2026-10-29
- Qwen3.8-27B EXL3 quant hits lowest KLD of 0.0002 in just 23 GB — suprjami · 2026-10-29
- Codex Pro user: 6.1 Terra keeps stopping mid-task, breaking cron-driven agent jobs — HunnyMal · 2026-10-29
- Leak: Anthropic's Fable 5.5 reportedly lands Nov 3 or Nov 10 — imjustnewatai · 2026-10-29