SimpleMemVLA tops four memory benchmarks by feeding raw timestamped visual history to VLA, no memory module
量子位 · wechat · 2026-09-18
ModelBest (OpenBMB) with HUST, RUC, Tsinghua, PKU and BAAI introduce SimpleMemVLA: instead of a dedicated memory module, it feeds timestamped, minutes-long raw visual history directly as context to a VLA.
- Results: SOTA overall on four memory benchmarks; on RoboMemArena (26 tasks, >1000 control steps per trajectory), a 126s history window lifts full-stage success to 63.6% (+17.4 pts), counting 31.4%→71.4%, occlusion 39.1%→64.3%.
- Key idea: 60s history is only 5.6k tokens vs a 262k context, so no lossy pre-compression. This avoids the "commitment at write time" problem of memory modules. Ablations show history acts via the hidden state passed to the action head.
- Visual in-context learning: with no parameter updates, splicing in new trajectory segments (moved occlusion spot, changed color cue) makes the model adapt, and only key history changes alter behavior.
- Streaming inference: KV-cache reuse overlapped with execution cuts 60s-history decision latency from 1.02s to 0.68s on one H100 (bf16), meeting a 0.96s real-time budget; 45min/245k-token history drops from 32.1s to 1.18s.
- Real dual-arm demo: robot uncovers red→green→blue blocks by color using pre-occlusion history, adapting paths across three layouts.
Paper, code and models are open-sourced.
More from Embodied
- CARLA driving expert with privileged maps trains vision-based imitation policy — abursuc · 2026-09-18
- AMP reframes robot manipulation as pixel classification to dodge action-space explosion — jiqizhixin · 2026-09-18
- Camera-Only Autonomous Drive Through the Pyrenees: Lessons From a Small PhD Team — abursuc · 2026-09-18
- ACE-Data-0 hits 120K downloads in first month: 17M-frame multimodal embodied AI dataset — liuziwei7 · 2026-09-18
- Dev fits a fly's 100,000-neuron brain into a $15 MCU to pilot micro-drones, open-sourced — alvelda · 2026-09-18
- Tesla Teams Arrive in China to Audit Suppliers for Optimus, Boosting Robot Stocks — abhiadesai · 2026-09-18