Un-fusing a realtime voice stack (STT → LLM → TTS) cut costs 14x — and the real win was text-level guardrails
Cloudsurfer_90 · reddit · 2026-09-22
After months in production, a developer ripped out their fused realtime voice model and rebuilt it as three stages: STT, then an LLM call, then TTS.
- Cost: dropped from $0.18/min to $0.0125/min — a 14x gap that dominates the business case at real call volumes.
- The tradeoff is latency: fused models are genuinely faster; their target is staying under 788ms voice-to-voice and some days they lose that fight. For live phone calls with talk-over, fused may still be worth it.
- The unexpected upside: with a split pipeline the LLM output is plain text before any audio exists, so every guardrail runs on text — you can catch hallucinated answers before a single audio sample is generated. The author says this inspectability matters more than the savings.
- Advice: start split if you're early and cost-sensitive or need to gate outputs; only fuse when latency truly kills you.
More from coding & agent
- Cloudflare launches Worker Previews: isolated production-like envs for every Git branch — iannuttall · 2026-09-22
- 'A Summer of AI Optimization': Veteran Engineer Reports Major Speedups in Widely Used Libraries — austinvhuang · 2026-09-22
- The 'chief of staff' pattern: how agentic coding scales past context windows — arpit_bhayani · 2026-09-22
- Jay Alammar and Team to Host Free AI Agents Masterclass on Sept 29 — JayAlammar · 2026-09-22
- Hands-on: testing Grok 4.7 coding in Cursor across 4 real projects with cost breakdown — Arindam_1729 · 2026-09-22
- Agent did nothing, reported success, and passed: why agent-generated evidence isn't evidence — Muted_Ad_9442 · 2026-09-22