VGGT-GS SLAM: Uncalibrated Monocular Gaussian Splatting SLAM with Feed-Forward Priors
Yuhang Han, Hao Wang, Jiaxi Cao, Xingyu Liu
cs.CV
2026-09-17
VGGT-GS SLAM jointly refines poses, Gaussians, and distortion from VGGT priors. Uncalibrated TUM ATE is 3.04 cm; dropping Gaussian alignment jumps to 7.20 cm.
Most Gaussian SLAM systems assume known intrinsics, and many want RGB-D. Phone and webcam video usually have neither. Feed-forward geometry models such as VGGT, DUSt3R, and MASt3R can emit pose and depth from RGB, but those predictions are an initialization: calibration error is absorbed into geometry, and independently built submaps need not share a scale.
VGGT-GS SLAM keeps VGGT-SLAM’s submap layout, turns the feed-forward output into optimizable Gaussian submaps, jointly refines poses, Gaussians, and shared intrinsics plus radial-tangential distortion inside each window, then uses covariance-aware Gaussian matching to fix sequential scale and to vet loop proposals.
Keyframes are windowed as 24 new frames plus one overlap. VGGT supplies extrinsics, per-frame intrinsics, and confidence-weighted depth. Depth is back-projected into Gaussians; shared intrinsics start from the window average and distortion from zero. Each submap runs 550 steps of differentiable bundle adjustment with a photometric term, a relative-depth loss with a dead zone, and an isotropy regularizer. The first pose is pinned as the local frame; VGGT depth is the scale prior.
Calibration gradients are analytic. CUDA kernels differentiate the projected 2D Gaussian mean and covariance with respect to focal length, principal point, and five distortion coefficients, so the rendering loss can update the lens model. Distortion is tanh-bounded, radial range 2 and tangential 0.05.
Gaussian-native alignment between submaps optimizes only log-scale; rotation is locked by the shared keyframe cameras. Matching uses learned covariances, not point centers. Loops are retrieved with SALAD, bridged by a joint VGGT inference, and accepted only if the matching loss and scale correction pass thresholds. Accepted measurements keep the original VGGT transform. A Sim(3) pose graph then updates submap-to-world transforms without merging maps.
Uncalibrated monocular indoor tracking:
| Dataset | This method ATE | Baseline |
| TUM-RGBD mean | 0.0304 m | AIM-SLAM 0.031 m; calibrated Splat-SLAM 0.030 m |
| 7-Scenes seq. 01 | 0.055 m | VGGT-SLAM++ 0.064 m (−14.1%) |
| ScanNet six-seq. mean | 0.090 m | VGGT-SLAM 2.0 0.103 m |
Novel-view synthesis averages 39.100 dB PSNR and 0.020 LPIPS on Replica, edging HI-SLAM2 at 39.046 / 0.029. On TUM, PSNR is 24.484 dB versus Splat-SLAM’s 24.413, with LPIPS 0.185 versus HI-SLAM2’s 0.197.
Gaussian-native alignment is the ablation that moves the needle: dropping it raises TUM ATE from 0.0304 to 0.0720 m. Disabling calibration optimization only reaches 0.0324 m. Across 95 Freiburg1 submaps, calibration cuts projection error from 14.791 to 11.185 px. Loop verification accepts 249 true loops with 2 false positives, against 247 / 7 for VGGT-SLAM 2.0. Full-pipeline speed is about 3.16 FPS; non-keyframe pose estimation is 69% of TUM runtime. The same backend still cuts ATE 22.7–58.9% when the frontend is swapped for DA3-Large or Fast3R.
This is a reproducible baseline for uncalibrated RGB, not another indoor Gaussian SLAM that assumes a calibration file. The portable pieces are analytic calibration Jacobians on the rendering chain, and covariance-aware Gaussian matching for scale and loop gating. 3 FPS is not real time. Localization is already in the same band as calibrated systems.
The authors flag sensitivity to prior quality and calibration observability, local map deformation, and window-induced latency. All numbers are indoor and short. The TUM gap over AIM-SLAM is 0.6 mm; the ranking is tight. Non-keyframe pose eating 69% of runtime is unfinished engineering. In a six-window control, adding distortion on top of intrinsics barely changed projection error.