VoiceMem: A Dual-Brain Memory System Cuts Voice AI Retrieval to 134ms
机器之心 · wechat · 2026-09-07
A team led by first author Zhifei Xie (NTU LV-Lab) and corresponding author Shuicheng Yan released VoiceMem, a streaming dual-brain memory framework for real-time voice interaction, jointly published by four top universities and OpenInteractionLab, with code and paper public.
It tackles two problems. Latency: existing memory systems take 2000ms per retrieval while voice dialogue allows only 400ms; VoiceMem splits retrieval into four parallel stages (listening, speechtail, anticipation, searching) compressed to 134ms, fully hidden inside the 300ms VAD silence window. It matches Mem0's Top-200 precision with only Top-10 on LongMemEval at 95% lower cost. Emotion: an "informative left brain" uses two-tier graph retrieval (compatible with Mem0/Zep) for facts, scoring 91.2 on LoCoMo with just 5 memories versus Mem0's 61.68 with hundreds; an "emotional right brain" maintains identity nodes and joint nodes that bind emotions and preferences to factual memories, lifting conflict/preference question accuracy from 10% to over 70%.
The team calls it the only memory system usable in real-time interaction today, aiming to be a real-time "Mem0". Qwen Audio Agent has integrated it as an optional memory system. Future work includes multimodal memory expansion, long-term forgetting/reinforcement mechanisms, and cheaper deployment. Xie previously led the Mini-Omni end-to-end speech dialogue model series.
Related event: VoiceMem Cuts Voice AI Memory Retrieval Latency to 134ms(2 posts)→
More from Models
- Fable 5.5 launch rumors persist for third week, but no frontier model drops this month — patience_cave · 2026-10-26
- OpenAI upgrades Decisions API: 400 questions per call, native voice and image support — OpenAIDevs · 2026-10-26
- Fangcun AI, China's first independent frontier AI safety lab, launches with DeepSeek eval report — JacquesThibs · 2026-10-26
- When AI hits an existential crisis: model confuses itself and everyone else — Ok-Delivery-7437 · 2026-10-26
- Kimi K3.1 Pro tops MathArena open models, closing frontier gap to about 6 months — teortaxesTex · 2026-10-26
- Ruff creator Charlie Marsh: frontier models find infinite bugs in any software — charliermarsh · 2026-10-26