Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction
Jiarong Han, Jincheng Xiong, Yuzhou Liu, Linzhe Shi, Changjie Wu, Ning Guo, Mu Xu, Hang Zhang, Ming Qian
cs.CV
2026-08-27
ABot-Recon caches 11 prior frames, predicts a current-camera point map and adjacent pose, then composes globally. Oxford Spires: 4.35 m ATE, 0.12° RPE-R (~40% down), 24.45 FPS.
Online recovery of camera motion and dense geometry from tens of thousands of frames has to run under a hard cap on memory and per-frame compute. Early streaming models already do causal, bounded-cost inference with a finite buffer or a recurrent state, then drift as the stream grows. LingBot-Map and HorizonStream stitch short-range context to persistent long-range memory, and historical-state management becomes the architecture.
Alibaba AMAP takes the opposite bet: keep the learned temporal state strictly local, and keep prediction targets independent of sequence length. If each step predicts a point map in the current camera frame and a relative pose to the previous frame, the model can train on short clips and run on 10k-frame streams. The global trajectory is a product of adjacent transforms. The regression target does not get harder as the first frame recedes.
ABot-Recon starts from public π³ weights and is adapted to causal streaming. Each new frame is encoded into patch tokens plus learned camera tokens, then processed with intra-frame attention and windowed causal cross-frame attention over 12 frames (the current frame and KV from the previous 11). A geometry head emits a current-camera point map and a confidence map. Adjacent camera tokens form a pairwise descriptor that predicts T{i-1←i}. KV outside the window is dropped, so temporal memory is O(K) and attention is O(NK).
Chaining relative poses still blows up rotation error. Two fixes, neither of which lengthens context.
A rotation refiner edits rotation only and leaves translation alone. Motion evidence comes from the pose descriptor. Visual evidence uses a coarse frame descriptor as a query into dense tokens. The two streams fuse, a gated TCN reads the last K pair features, and an axis-angle residual is composed onto the initial relative rotation. The module adds 4.75M parameters, about 0.48% of the full model.
Supervision is not adjacent-only. Every pair inside the window with gap at most K-1 has its chained pose compared to ground truth, with larger gaps weighted more (exponent γ in (0, 1)). The model sees composition error during training, not only pairwise error. Point-map, normal, confidence, and residual-smoothness terms sit on top.
Training has two stages. Stage I uses 32-frame clips for 32K steps to stabilize local geometry and adjacent motion. Stage II stretches clips to 128 frames for 38K steps and turns the refiner on. A final 4K-step pass freezes the backbone and calibrates confidence. Data mix is 30 synthetic and real sets, sampled about 62% / 38%. At inference, an optional loop-closure backend (DINOv2-SALAD retrieval into a pose graph) can be bolted on without retraining.
Three long-horizon pose benchmarks, all stride 1, Umeyama Sim(3) alignment, then ATE and RPE. ATE is absolute trajectory RMSE; RPE is relative pose error.
| Method | KITTI ATE (m) | Oxford ATE (m) | Oxford RPE-R (°) | VBR ATE (m) |
| HorizonStream | 22.51 | 8.81 | 0.20 | 29.02 |
| LingBot-Map | 29.13 | 7.32 | 2.29 | 29.45 |
| ABot-Recon | 18.25 | 4.35 | 0.12 | 30.14 |
| loop closure | 13.49 | 4.02 | 0.12 | 9.99 |
On Oxford Spires, ATE and RPE-R drop about 40% versus the previous best streaming numbers. On VBR without loop closure, ATE is slightly worse than HorizonStream (30.14 vs 29.02); with loop closure it falls to 9.99, below HorizonStream’s 17.64. On KITTI-02, H100 frame-by-frame inference runs at 24.45 FPS and 6.71 GB, versus HorizonStream’s 8.02 FPS / 13.04 GB.
Dense reconstruction on Oxford Spires (4 m threshold) reaches Chamfer 1.37 and F1 91.81, ahead of LingBot-Map at 1.68 / 90.58. Indoor 7Scenes and TUM-Dynamic are not a win: 7Scenes F1 is 94.88, under LingBot-Map’s 96.01 and HorizonStream’s 98.22.
Ablation ATE on KITTI / Oxford / VBR: 32-frame adjacent-only baseline 56.60 / 10.16 / 52.90; add composition loss 27.66 / 9.40 / 44.72; 128-frame training 22.31 / 5.95 / 36.77; add the rotation refiner 18.25 / 4.35 / 30.14. Longer clips cut ATE another 19.3%, 36.7%, and 17.8%; the refiner cuts 18.2%, 26.9%, and 18.0%.
Recent streaming reconstruction treats long-range memory as mandatory. A 12-frame KV cache that beats those systems on kilometre-scale driving and landmark sequences says the bottleneck is often how you parameterize the prediction, not how much history you store. For onboard mapping, constant VRAM plus optional loop closure is a cleaner split than welding hierarchical memory into the net. Persistent geometric context still helps in compact indoor scenes; the paper says so, and this is not a wholesale SLAM replacement.
The discussion is clear: on tight indoor sequences with frequent revisits, reconstruction is not uniformly best, because the method refuses scene-level state. Dedicated dynamics handling, external memory, and larger pretraining are listed as future work.
Read the table, not the slogan. VBR without loop closure does not beat HorizonStream on ATE; “local context is enough” is mainly an Oxford and KITTI claim. DPVO and DPV-SLAM post lower Oxford ATE (1.56 / 2.95) but need intrinsics; ABot-Recon is RGB-only, and that caveat cuts both ways. Training includes internal synthetic and internal real data, so the mix is not reproducible from the paper alone. The refiner touches rotation only; fast pure translation and scale drift sit outside its reach. Loop closure, a standard retrieval-plus-pose-graph add-on, still helps a lot, which means the pure local chain still drifts, just more slowly than older streaming models.