SimpleMemVLA tops four memory benchmarks by feeding raw timestamped visual history to VLA, no memory module
量子位 · wechat · 2026-09-18
ModelBest (OpenBMB) with HUST, RUC, Tsinghua, PKU and BAAI introduce SimpleMemVLA: instead of a dedicated memory module, it feeds timestamped, minutes-long raw visual history directly as context to a VLA.
- Results: SOTA overall on four memory benchmarks; on RoboMemArena (26 tasks, >1000 control steps per trajectory), a 126s history window lifts full-stage success to 63.6% (+17.4 pts), counting 31.4%→71.4%, occlusion 39.1%→64.3%.
- Key idea: 60s history is only 5.6k tokens vs a 262k context, so no lossy pre-compression. This avoids the "commitment at write time" problem of memory modules. Ablations show history acts via the hidden state passed to the action head.
- Visual in-context learning: with no parameter updates, splicing in new trajectory segments (moved occlusion spot, changed color cue) makes the model adapt, and only key history changes alter behavior.
- Streaming inference: KV-cache reuse overlapped with execution cuts 60s-history decision latency from 1.02s to 0.68s on one H100 (bf16), meeting a 0.96s real-time budget; 45min/245k-token history drops from 32.1s to 1.18s.
- Real dual-arm demo: robot uncovers red→green→blue blocks by color using pre-occlusion history, adapting paths across three layouts.
Paper, code and models are open-sourced.
More from Embodied
- Baidu's Apollo Go wraps two-week public robotaxi test rides in Tung Chung, Hong Kong — Baidu_Inc · 2026-11-04
- imec neural probe integrates photonics directly into 130nm CMOS backend — jwt0625 · 2026-11-04
- Chris Paxton: interactive robot sim is the future but glaringly data-limited — Scobleizer · 2026-11-04
- DreamX: video diffusion models as universal latent action learners for robots — kaist-ai · 2026-11-04
- RoboTryOn turns human videos into robot demos, lifting success rate from 2% to 60% — kaist-ai · 2026-11-04
- INMO Future glasses: dual 1080p displays, 125g, $799 early-bird on Kickstarter Dec 10 — Scobleizer · 2026-11-04