Running 20 concurrent low-latency regional-language voice agents on a single L4 GPU
DistrictUnique5259 · reddit · 2026-10-18
A team shares their production voice-agent stack for Malayalam: self-hosted Pipecat for orchestration, Mai Transcribe 2 for STT, a Sarvam conversational LLM with objection handling and human fallback, and a fine-tuned VoxCPM2 TTS squeezed to run 20 concurrent streams on a single L4 GPU without latency spikes — the hardest part. As a Meta Tech Partner they wired the agent to the WhatsApp API so callers get an automated quotation before hanging up. Indian pricing keeps margins thin, so the author is seeking sales partners in the US/UK/Gulf for a 50-50 split, and is open to technical Q&A on the setup.
More from coding & agent
- Building a multi-agent shared knowledge base with two GitHub repos — and one memory pitfall — gregbarbosa · 2026-10-18
- Fully agentic MV pipeline: Claude + MiniMax H3 renders 28 scenes with 0ms sync drift — Cheap_Credit_3957 · 2026-10-18
- Dev builds seamless laptop/web/mobile dev env with Claude Code self-hosted runners + Tailscale — davidcrawshaw · 2026-10-18
- SOC Prime launches LogTotal: rule-first log analysis with AI interpretation at the end — cyb3rops · 2026-10-18
- Thorsten Ball: A 'wrong' code push matters far less now that AI makes changes cheap — sloppenheimer · 2026-10-18
- Every trains a Slack agent to edit like its editor-in-chief using a custom skill — every · 2026-10-18