Low-latency game AI voice explained: caching, pregen audio and manual MTP hit ~2s response
karminski3 · x · 2026-10-30
Blogger karminski3 breaks down how to get 2s latency from LLM game teammates: pre-record high-frequency lines ("enemy on the right") instead of calling the model; hash-cache generated TTS audio for reuse; hand-roll MTP by prompting the model to output only {"decision":n} from a candidate list, exploiting fast prefill/slow decode; and use human attention limits as a fallback — give the model a 2s timeout, and have another AI teammate banter to cover a 5s late reply. A depth-5 FIFO queue prevents talking over each other. He also reviews Kimi-K3.1-Pro: best-in-class Chinese front-end/3D skills and stronger agent goal-focus, but weaker back-end agentic coding than K3 due to pessimistic self-imposed timeouts — recommend plan-first workflows.
More from coding & agent
- Autonomous agents debated his sleep rule overnight, then called him at 7am to save a release — airesearch12 · 2026-10-30
- Dev's slangy commit message 'api key ui very nice' becomes a self-own screenshot — itsOmSarraf_ · 2026-10-30
- lattedit: a local-first browser LaTeX editor with a JSON API for AI agents — cneuralnetwork · 2026-10-30
- Pedro Domingos: COBOL will outlive Claude Code — pmddomingos · 2026-10-30
- Anthropic's Felix Rieseberg: 4 ways to use AI more ambitiously — speed, altitude, depth, range — FinanceYF5 · 2026-10-30
- kalomaze: agents keep sneaking MSE regression losses into multimodally conditional ML problems — kalomaze · 2026-10-30