BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
Peiyan Li, Yuze Zhu, Yixiang Chen, Qisen Ma, Yuan Xu, Jiabing Yang, He Guan, Yan Huang, Hongtao Wu, Xiao Ma, Tao Kong, Liang Wang, Tieniu Tan
cs.RO
2026-08-06
BridgeVLA++ adds a spatio-temporal memory to BridgeVLA's view-and-heatmap 3D policy: memory benchmarks jump 18.9% to 96.0% (RMBench) and reach 99.7% (MemoryBench) at no cost to spatial tasks, and RLBench rises to 93.7%.
Teaching robots to perform 3D manipulation (grasp, place, push, insert) from a language instruction, mainstream vision-language-action (VLA) models share three chronic weaknesses: they are data-hungry, they break under distribution shift, and they have no memory, acting only on the current frame. Memorylessness dooms two families of tasks: long-horizon ones that require remembering which sub-goal was just reached (say, pressing a sequence of buttons in order), and ones where the arm occludes the target and the policy must lean on geometry it saw earlier.
The team's earlier BridgeVLA had already fixed the first two weaknesses, by keeping both the input and the output of the 3D problem in the native 2D image space of a pretrained vision-language model (VLM). But BridgeVLA was still a strictly memory-free policy, predicting each action from the current observation alone. BridgeVLA++ adds a unified spatio-temporal memory on top of it, gaining memory without surrendering the original data efficiency and generalization.
BridgeVLA's recipe first, since the memory is grafted onto it. On the input side, a colored point cloud is rendered into three orthographic projection views (top, front, right); the VLM (PaliGemma here) sees only images, and even proprioceptive state is withheld, which preserves the image-text alignment from pre-training. On the output side, rather than autoregressively predicting actions as a token sequence, the model predicts one 2D translational heatmap per view; candidate points are aggregated across the three views and the argmax is back-projected into a 3D end-effector translation, with rotation and gripper state filled in by a small MLP. The pipeline is coarse-to-fine: a first pass predicts a coarse waypoint, the point cloud is cropped and magnified around it, and a second pass refines. A 2D-heatmap grounding stage on the 120K RoboPoint localization split pre-trains the heatmap prediction.
The memory lives in the VLM's visual-token space, in two branches. Temporal memory sits at the coarse stage and answers "what to do next". It caches three kinds of tokens: the three anchor views of the initial observation (a fixed reference to detect scene change), the two most recent keyframes, and milestone keyframes selected by a learnable adaptive sub-goal gate, a small MLP that predicts a retention probability per frame to suppress redundancy. Spatial memory sits at the fine stage and answers "where exactly to act", targeting occlusion. It stores the initial point cloud and, once the coarse stage emits a waypoint, re-renders that initial cloud under the same crop-and-zoom, letting each current view attend only to its matching reference view to recover geometry now hidden.
The division of labor is deliberate: the coarse stage sees the whole workspace and picks the next region, so temporal memory goes there; the fine stage is where occlusion bites during precise localization, so spatial memory goes there. All three memory components are injected by cross-attention, with current visual tokens as queries and memory tokens as keys/values, adding only 270M parameters (9.2% of the 2.92B backbone) and lifting per-step latency from 0.35s to 0.57s on a single RTX 4090.
On the spatial benchmarks, memory is almost a free win. On RLBench, BridgeVLA++ averages 93.7% success, above BridgeVLA's 90.5% and 6.9 points over the prior SOTA SAM2Act (86.8%). On COLOSSEUM (14 out-of-distribution perturbations) it averages 65.2%, slightly above BridgeVLA, and under the "all perturbations at once" setting reaches 38.9%, more than double BridgeVLA's 18.7%. On GemBench it averages 51.1%, slightly above the prior best 3D-LOTUS++ (48.0%).
The two memory benchmarks are where the gap opens. On RMBench (dual-arm, nine tasks), the memory-free BridgeVLA scores only 18.9%; BridgeVLA++ reaches 96.0%, 13 points above the prior SOTA MemoryWAM (83.0%), and on the trial-and-error sorting task Battery Try it is 96% versus 41%. On MemoryBench (single-arm), BridgeVLA scores 11.3%; BridgeVLA++ reaches 99.7%, with every seed at or above 99.3%.
| Benchmark | BridgeVLA (memory-free) | BridgeVLA++ | Prior SOTA |
| RLBench (spatial) | 90.5 | 93.7 | 86.8 (SAM2Act) |
| RMBench (memory, dual-arm) | 18.9 | 96.0 | 83.0 (MemoryWAM) |
| MemoryBench (memory, single-arm) | 11.3 | 99.7 | 94.3 (SAM2Act+) |
Real robots confirm it. On the Dobot dual-arm, the average success on three memory-dependent tasks rises from BridgeVLA's 20.0% to 93.3%, three times SAM2Act+. Data efficiency remains a BridgeVLA strength: on the real Franka, BridgeVLA hits 95.4% with only 3 demonstrations per task, while π0.5, which shares the same PaliGemma backbone, manages 20% even with 10. The sample efficiency comes from the input-output alignment design, not the backbone.
The ablations confirm the division of labor. Removing temporal memory drops RMBench from 96.0% to 21.3% (essentially back to the memory-free base), so memory tasks ride on temporal memory. Removing spatial memory barely moves RMBench but gives targeted lifts on occlusion-heavy RLBench tasks such as Sort Shape. And removing BridgeVLA's heatmap decoding collapses RLBench from 90.5% to 31.4%, confirming the heatmap output design is the foundation.
For robot-manipulation practitioners, BridgeVLA++ delivers three things at once: data efficiency, robustness to distribution shift, and memory. These have usually been pursued separately, and this work reaches SOTA on memory benchmarks without regressing spatial tasks, showing memory can be added "for free".
The most transferable idea at the method level is keeping memory inside the VLM's visual-token space instead of bolting on a separate external memory module. The action interface (heatmaps) is untouched, injection is plain cross-attention, and the cost is about one-tenth more parameters and latency. The learnable sub-goal gate is also worth borrowing: rather than indiscriminately storing every step, learn a gate to pick milestone frames so near-duplicate ones do not dilute the critical history.
There is no dedicated limitations section; a few admissions are scattered through the paper. The clearest is category generalization: on the real Franka with unseen object categories, the policy sometimes ignores the target object and moves straight to the destination. The authors attribute this to the 2D pre-training images being mostly third-person views unlike the orthographic robot renders, and to object-level grounding supervision where manipulation keypoints need not lie on an object.
The sub-goal gate currently relies on per-segment language annotations to train (the last keyframe of each segment is the positive), so on benchmarks without such annotations it is disabled and only the two neighboring keyframes remain; replacing it with a self-supervised alternative is listed as future work. On the long-horizon GemBench L4 split, every end-to-end baseline is near zero and BridgeVLA++ reaches only 8.2%, far from solving long-horizon. RMBench is a single 100-episode evaluation with the best checkpoint per task, without the multi-seed averaging used elsewhere, so its variance is not reported. And the tables do not directly compare against common 2D VLAs such as RT-1, OpenVLA, or 3D-VLA (they appear only in related work); anyone wanting that cross-comparison has to add it.