AssemblyAI open-sources voice agent LLM benchmark across 50+ models
AssemblyAI · x · 2026-10-14
AssemblyAI released an open-source benchmark evaluating 50+ LLMs for voice agents, all accessible via LLM Gateway with an existing AssemblyAI API key.
- The benchmark is open source, so you can rerun it on your own transcripts
- Speed matters for voice agents: on 30-second and 2-minute inputs, the fastest open models answer in under a second (median), making open models the quickest in the run
- Every failure counts against quality: wrong format, refusal, timeout, and errors all hurt the score
Related event: AssemblyAI Launches Voice Standard Bench for 50+ Models(5 posts)→
More from Research
- CMU paper: a 4B model rewrites an agent's harness and beats the 35B model it learned from — irinarish · 2026-10-14
- First blind patient in PRIMA implant trial reads a word after two rehab sessions — juanbenet · 2026-10-14
- Prefix Sliding paper: training-free 3x speedup for long reasoning via prefix + sliding window — cephaloform · 2026-10-14
- Formalized proof completed; author suggests using AI to explore proof strategies — tak3sh8 · 2026-10-14
- LLM agents beat top numerical optimizers in 4 of 5 black-box optimization tasks — lulzxdxdxd · 2026-10-14
- Steven Strogatz on NPR: AI is steamrolling through math — stevenstrogatz · 2026-10-14