HeLM: explicit language memory gives VLA robots +12.8 points on non-Markovian manipulation
SeoulNationalUniv · hf · 2026-11-06
Seoul National University proposes HeLM (Hierarchical Explicit Language Memory), a hierarchical VLA framework tackling the Markovian assumption of most VLA models — mapping only the current observation to actions, which fails in non-Markovian environments needing working memory. Inspired by humans' inner monologue, HeLM maintains task-relevant history in explicit language via factorized event detection and event-triggered memory updates. Across 10 non-Markovian real-world tasks (56 variants, including five new inter-episode tasks), it improves intra-episode success by 12.8 points over the strongest baseline, hits 78% subtask success on compositional generalization, and uses up to 4.5× less memory while running 2.5× faster than the keyframe baseline at long horizons.
More from Embodied
- Lightwheel and Synopsys bring CAE-grade contact physics into robot simulation — lukas_m_ziegler · 2026-11-06
- Harvard-led team's MFM memory system lets frozen-weight agents keep learning in Minecraft — du_yilun · 2026-11-06
- RAX lets coding agents unscrew a real jar with dexterous hands: 6 chained skills, 140° rotation — du_yilun · 2026-11-06
- 3 Years and Tens of Thousands of Lines of Code vs. One Prompt: AI Scene Generation's Leap — liliyu_lili · 2026-11-06
- UWaterloo 3D prints Invar36 flexures with directly mounted interferometer optics — jwt0625 · 2026-11-06
- LeRobot v0.7.0 ships as ecosystem passes 50,000 developers and 300+ organizations — LysandreJik · 2026-11-06