AWS tutorial: streaming Kokoro-82M TTS on SageMaker lets voice agents speak from sentence one
AWS ML Blog · rss · 2026-10-23
An AWS ML Blog engineering tutorial tackles the "5 seconds of silence" problem in voice agents: each sentence streams to a Kokoro-82M model (82M params, one mid-tier GPU) hosted on a SageMaker AI real-time endpoint as soon as the agent produces it, so audio playback starts immediately. Key details:
- A custom container exposes POST /invocations and a bidirectional-stream WebSocket on ml.g5.xlarge, with streaming capability declared via Docker label; AgentCore runtime hosts the Strands agent.
- Pitfall 1: SageMaker's bidirectional transport delivers binary WebSocket frames only; Starlette's receivetext() throws KeyError('text') with a hard-to-debug error frame — handle text and bytes explicitly.
- Pitfall 2: keep one persistent session across sentences instead of closing per response, or reconnection cost erases the latency win.
- Includes a reusable IncrementalSentenceChunker that emits complete sentences from a live token stream, with flush() for text lacking final punctuation.
More from coding & agent
- Claude Code Pushes Urgent Security Update Across All Installations — umeshonai · 2026-10-23
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- 3 flight-booking rules my AI agent checks that I'd never do by hand — Efistoffeles · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23