NVIDIA paper: routing voice-agent tool calls to a text LLM lifts recall above 92%

omarsar0 · x · 2026-09-20

A new NVIDIA research paper gives full-duplex speech models tool-calling ability by routing decisions out of the speech pipeline:

• Commercial duplex voice models complete only 31-51% of grounded customer-service tasks under clean conditions, while text agents like GPT-5 hit 85% on the same tasks.
• The fix: the duplex frontend learns to emit a delegation token, forwards streaming transcripts to a text backend LLM for the tool call, and receives results via a lightweight prefill-and-repeat mechanism before streaming TTS speaks them.
• Results: 92.0-97.2% tool-call recall, 81.2% accuracy at rejecting irrelevant calls, with turn-taking, streaming ASR WER, and spoken-language intelligence unchanged.

Original post →

More from Research

• Microsoft's StudentSim uses error-prone simulated students to train better AI tutors — The Decoder · 2026-09-20
• Sandberg's old paper warned lab automation would change the risk chain — now AI actors enter it — anderssandberg · 2026-09-20
• Da7em Bench: independent AI benchmark scores models on 200 real client tasks across 12 areas — airesearch12 · 2026-09-20
• Researcher Accuses Laya Author of Overstating Work as Jev-Like System — Paimaamu · 2026-09-20
• LLM vs classical ML across 8 datasets: labeled data still favors SVM and XGBoost — Ok_Juggernaut2187 · 2026-09-20
• Claude Speeds Up 30+ Open-Source Biomolecular Models 4x on Average, All Code Open-Sourced — FinanceYF5 · 2026-09-20

Research channel →