LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction
Lin-Zhuo Chen, Jian Gao, Shangzhan Zhang, Yihang Chen, Ka Leong Cheng, Yipengjing Sun, Liangxiao Hu, Nan Xue, Xing Zhu, Yujun Shen, Yao Yao, Yinghao Xu
cs.CV
2026-04-16
LingBot-Map turns SLAM-style anchor, window, and trajectory memory into attention, runs about 20 FPS at 518x378, and cuts Oxford Spires ATE to 5.37 versus 12.87 for offline DA3.
Feed-forward 3D foundation models such as VGGT and Depth Anything 3 already predict camera pose and dense point maps in one pass. They do it offline: the full image set is present, and bidirectional attention sees everything. Streaming is a different constraint. CUT3R compresses history into a recurrent state and forgets geometric priors. StreamVGGT and Stream3R cache nearly the entire history with causal attention, so memory and compute grow with sequence length. Hybrid systems such as VGGT-SLAM keep keyframes and pose graphs, but the selection rules are hand-crafted and iterative optimization is hard to run in real time.
The design question is which geometric context to keep, not how much history to store. A 10,000-frame video cannot retain every image token, and it also cannot collapse the past into a single blurry state.
LingBot-Map turns three SLAM-style memories into learned attention, called Geometric Context Attention (GCA).
The backbone is DINOv2-Large with 24 alternating frame-attention and GCA blocks. A camera head predicts absolute pose; a depth head predicts depth. Training is two-stage: a global-attention base model on 2–24 views (about 160K steps on 64 GPUs, roughly 21,500 GPU-hours), then GCA with a view curriculum from 24 to 320. Default inference is Direct mode with a FlashInfer paged KV cache, about 20 FPS at 518×378. Sequences beyond roughly 3,000 frames switch to VO mode and Sim(3)-align overlapping windows.
Oxford Spires, sparse 320-frame setting:
| Method | Type | AUC@15 | ATE |
| DA3 | offline | 49.84 | 12.87 |
| VIPE | optim | 45.35 | 10.52 |
| CUT3R | online | 5.98 | 18.16 |
| LingBot-Map | online | 63.22 | 5.37 |
On the dense 3,840-frame split, ATE moves only from 5.37 to 5.60; CUT3R jumps from 18.16 to 32.47. ATE on ETH3D / 7-Scenes / Tanks and Temples is 0.43 / 0.08 / 0.21. Reconstruction F1 is 86.80 on ETH3D (Wint3R 77.28), 82.38 on 7-Scenes, and 65.10 on NRGBD. Sintel ATE is 0.10 and KITTI is 24.12, against VGGT-Long at 0.17 and 31.89.
Video RoPE is the largest ablation win on ATE (7.46 to 5.98). A window of 64 versus full causal attention runs at 20.29 vs 11.87 FPS, 13.28 vs 36.06 GB, and better ATE (5.98 vs 6.60). Default k=64, n=8 is 20 FPS on an H800 and 12 FPS on an RTX 4090.
The SLAM state split is written into the attention mask, not bolted on as bundle adjustment. For robotics, AR, and any system that must map while moving, a feed-forward pass with nearly constant per-frame cost on 10k-frame streams is closer to deployable than offline reconstruct-then-align. The result also says the bottleneck for streaming 3D foundation models is which image tokens to drop, not another layer of global attention.
The compute bill is foundation-model scale: about three weeks on 64 GPUs. This is not a small plugin.
The paper lists three. There is no explicit loop closure, so revisits rely on soft correction from trajectory memory. There is no test-time optimization, so hard scenes get no BA cleanup. Direct mode degrades after about 3,000 frames; longer inputs use VO mode, and Sim(3) joins at window boundaries add another drift term.
Local translation is still worse than iterative BA: VIPE RPE-trans is 0.43 against 0.93. KITTI ATE of 24.12 is large in absolute terms on city-scale driving; the baselines are simply worse. Dynamic objects, distortion-aware cameras, and LiDAR/IMU fusion are left as future work. Ablations ran only on TartanGround, at about 3,840 GPU-hours each, and were never repeated on Oxford Spires.