Dev building two-way call translation on Gemini Live API shares voice-pinning, latency and accuracy pain points
MaleficentMention519 · reddit · 2026-10-10
A developer building WireLang, a desktop app that translates call/video audio in real time via gemini-3.5-live-translate-preview, is beta-testing a Headset mode for two-way call translation: each side speaks their own language and hears the other translated, no install needed on the far end, billed through the user's own Gemini API key.
Running two Live sessions on real calls surfaced three open problems:
- Voice consistency: the model picks a voice per connection, so reconnects can swap voices mid-call. His fix: pin a prebuilt voice via speechconfig.voiceconfig.prebuiltvoiceconfig.voicename inside generationconfig (top-level placement is rejected), plus session resumption. Open questions: accent drift across turns, whether language codes like en-US vs en-GB affect accent.
- Latency: steady state is under 2s, but first audio takes 4s. Current mitigations: tuned VAD (high sensitivity, 500ms silence, 200ms prefix padding), session resumption with context-window compression. He's asking about session pre-warming, VAD tuning, and chunk-size effects.
- Accuracy: occasional meaning flips on short phrases ("won" vs "lost"), with background noise as the biggest degrader — and the translate config accepts only a target language, with no source hint or instructions. He's exploring transcription-config tweaks and a second-pass text-model re-translation for captions.
Beyond the questions, the post doubles as a practical field report on Live API specifics: voice pinning config, VAD parameters, and resumption behavior.
More from coding & agent
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- 3 flight-booking rules my AI agent checks that I'd never do by hand — Efistoffeles · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- How do you audit AI agent decisions in regulated workflows? A field survey — Ok_Way_1993 · 2026-10-23