LiveKit Optimizes Gemma 4 31B for Voice Inference
xiaohu · x · 2026-07-17
LiveKit 在自家推理平台 LiveKit Inference 上跑 Google 的 Gemma 4 31B，并针对实时语音场景做了优化。
对比 GPT-4.1，它的首字延迟从 1006ms 降到 192ms，接话延迟快约 5.2 倍，成本约便宜 83%。在酒店前台任务测试里，完成率达到 88%，高于 GPT-4.1 的 73% 和 Gemini 2.5 Flash 的 64%。
Related event: LiveKit Optimizes Gemma 4 31B for Voice Agents(4 posts)→
More from Infra
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- NVIDIA claims HGX Rubin NVL8 delivers 10x tokens per megawatt vs B200 — thursdai_pod · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Together lets multiple LoRA post-training experiments share the same GPU capacity — togethercompute · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27