Qwen3-TTS on one H100: sub-50ms p95 latency at ~$2 per 1M characters
bibryam · x · 2026-08-31
Nari Labs released their own serving implementation and benchmark for Qwen3-TTS 1.7B CustomVoice: on a single NVIDIA H100 SXM it hits 10 RPS with sub-50 ms p95 time-to-first-audio and zero underruns during real-time playback.
- Compared against vLLM-Omni, SGLang-Omni, VoxServe, and M under 5-minute Poisson open-loop traffic, theirs is the only implementation achieving sub-50 ms p95 TTFA, staying under 100 ms even at 20 RPS.
- Cost: 630 chars/sec at 10 RPS; at $4.29/hr for the H100 that's roughly $2 per 1M characters — versus $100/1M for ElevenLabs V3 and $49/1M for Cartesia Sonic 3.5 at higher latency.
- Implementation and benchmark are open source; they also define the four requirements of "real-time" TTS: low audible TTFA, zero underruns, capacity as RPS scales, non-malformed output.
More from Infra
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16