VoiceMem: A Dual-Brain Memory System Cuts Voice AI Retrieval to 134ms
机器之心 · wechat · 2026-09-07
A team led by first author Zhifei Xie (NTU LV-Lab) and corresponding author Shuicheng Yan released VoiceMem, a streaming dual-brain memory framework for real-time voice interaction, jointly published by four top universities and OpenInteractionLab, with code and paper public.
It tackles two problems. Latency: existing memory systems take 2000ms per retrieval while voice dialogue allows only 400ms; VoiceMem splits retrieval into four parallel stages (listening, speechtail, anticipation, searching) compressed to 134ms, fully hidden inside the 300ms VAD silence window. It matches Mem0's Top-200 precision with only Top-10 on LongMemEval at 95% lower cost. Emotion: an "informative left brain" uses two-tier graph retrieval (compatible with Mem0/Zep) for facts, scoring 91.2 on LoCoMo with just 5 memories versus Mem0's 61.68 with hundreds; an "emotional right brain" maintains identity nodes and joint nodes that bind emotions and preferences to factual memories, lifting conflict/preference question accuracy from 10% to over 70%.
The team calls it the only memory system usable in real-time interaction today, aiming to be a real-time "Mem0". Qwen Audio Agent has integrated it as an optional memory system. Future work includes multimodal memory expansion, long-term forgetting/reinforcement mechanisms, and cheaper deployment. Xie previously led the Mini-Omni end-to-end speech dialogue model series.
Related event: VoiceMem Cuts Voice AI Memory Retrieval Latency to 134ms(2 posts)→
More from Models
- SimpleBench results show AI models beating humans on common sense — DigSignificant1419 · 2026-09-07
- Why is nobody talking about Tencent Hy4, the most-used model on OpenRouter? — cantor8 · 2026-09-07
- Anthropic says Claude wrote the longest math proof ever, cracking a 358-year-old problem — basedjensen · 2026-09-07
- Similarweb: ChatGPT's AI traffic share falls from 73.3% to 55.5% in 12 months — gaganghotra_ · 2026-09-07
- GPT-6 Astra (and Pro?) spotted on Simple-Bench leaderboard — From_Internets · 2026-09-07
- Testing Gemini as music understanders: Pro 3.1 solid, Flash models hallucinate sounds — teropa · 2026-09-07