Full voice agent (STT+LLM+TTS) on one A40: 1.28s median latency, tuning details shared
Broad_Breakfast_1172 · reddit · 2026-10-26
- Measured latency over 73 spoken replies: end-of-speech pause 200ms, STT 271ms, LLM first sentence 372ms, TTS first audio 298ms — total median 1.28s; web-search replies take 2–4s, pushing overall P90 to 3.9s.
- Gotchas documented: Gemma emits an empty thinking block after tool calls; Qwen's chat template rejects mid-conversation system messages; a WebSocket client bug dropped all audio after the first interruption.
- Try it: interrupt mid-sentence, ask Tokyo time (real clock tool), query a fact sheet where it admits unknowns, or type instead of talking.
Public demo available (5-minute visits, no voice recorded, text kept 30 days). Author invites others to share time-to-first-audio numbers with open models.
More from coding & agent
- Resin: a runtime that lets agents write forward passes to build standalone LLM inference engines — andersonbcdefg · 2026-10-26
- unified-brain: OSS SQLITE memory layer claims frontier-class coding at 1/18th cost — StatusFoundation5472 · 2026-10-26
- Dev shares multi-agent coding stack: routing tasks to Opus, Fable and Astra by code size — sh_reya · 2026-10-26
- App Store Connect CLI 5.17.0 ships one-call screenshot reordering, 8k stars on GitHub — rudrank · 2026-10-26
- Dev shows GPT-6 Astra Ultra coding for 41+ hours straight on a Dreamcast game — AIandDesign · 2026-10-26
- Bitpanda crypto exchange gets an MCP server for trades, wallets and transactions — modelcontextprotocol · 2026-10-26