ZJU team's EMem-Bench: 2,554 samples test whether embodied agents really remember
新智元 · wechat · 2026-10-18
Zhejiang University's OmniAI team released EMem-Bench, an executable benchmark for long-term embodied memory, plus the EMem memory system and EMem-8B model (30K HF downloads in three weeks):
- Failure analysis: of 100 failure traces from Gemini-3-Flash and Qwen3-VL-32B, 91% were memory-related — missed visual cues (35%), stale environment state (18%), unrecorded interaction feedback (31%), poor experience transfer (7%).
- Benchmark: four task types (passive observation, dynamic tracking, interaction failure, experience generalization) built on AI2-THOR/ProcTHOR; 1,118 scenes, median histories of 78-111 steps, 2,554 verified samples.
- Results: Gemini-3-Flash leads full-context at 64.2% average success; with a frozen GPT-5.4-mini backbone, EMem lifts success from 35.3% to 58.9% (+23.6 pts), far above rivals like TeleMem; EMem-8B reaches 46.0% (+20.6 pts over Qwen3-VL-8B).
- Key finding: stuffing more history images into context hurts — success drops from 36.6% to 26.2% while tokens grow 4.9x. Ablations confirm scene/spatial/episodic memories are complementary.
More from Embodied
- Tesla owner with 26,000 km in one year explains why he won't buy FSD — michalmalewicz · 2026-10-18
- Mind Children's Codey: a child-sized companion humanoid for elders, pilots in 2027 — CyberRobooo · 2026-10-18
- SharpaDex v1.0: 300 hours, 28,993 bimanual dexterous demos with fingertip tactile sensing, open-sourced — DJiafei · 2026-10-18
- SNU Visual Computing Lab shares research overview: teaching robot hands human-level dexterity — CSProfKGD · 2026-10-18
- Hands-on: Surface Laptop Ultra + NVIDIA RTX Spark tested for local AI — draginol · 2026-10-18
- Chery-backed AiMOGA opens global pre-orders for Mobo home humanoid robot — emmanuelvivier · 2026-10-18