BayesianGS-SLAM: Uncertainty-Aware Neural Rendering SLAM via Probabilistic Formulation
Kyeongsu Kang, Seongbo Ha, Sibaek Lee, Hyeonwoo Yu
cs.RO
2026-09-21
Sungkyunkwan University builds a tractable RGB-D predictive variance from sensor noise and opacity, then reuses it for mapping, residual-normalized tracking, and surprise-based keyframes, reaching 3.12 cm TUM ATE with about 70% fewer keyframes.
Neural-rendering SLAM tracks and maps from rendered RGB-D residuals. How much those residuals can be trusted changes with sensor noise, coverage, and holes in the map. Without a reliability estimate, bad residuals drag the pose, and frames the map already explains still trigger a mapping step.
Classical SLAM is Bayesian. Neural-rendering systems usually keep uncertainty inside mapping, model color only (VarSplat), or pay for sampling and ensembles that do not fit online. This paper wants one predictive uncertainty for both color and depth that mapping, tracking, and keyframe selection can share.
The backbone is LoopSplat's RGB-D 3DGS SLAM. Uncertainty splits in two: sensor noise from a small conv/MLP on the current RGB-D frame, and map uncertainty on Gaussian opacity, because opacity controls visibility and front-to-back weights for both color and depth.
Rendering is nonlinear, so exact propagation is intractable. The paper linearizes alpha compositing in opacity. Rendered variance is the sum of (ci − rendered color)² Ti² σα² along the ray, and the same for depth. Predictive variance is sensor plus that term. Mapping adds a Gaussian NLL on color and depth: residual over variance, plus log variance so the network cannot shout uncertainty to infinity.
During tracking the Gaussians and the uncertainty net stay frozen. Full NLL with the log term is unstable online, so the tracker uses uncertainty-normalized L1: residual over standard deviation. Keyframes reuse the same predictive NLL as a surprise score, with a fallback every N=5 frames. The implemented threshold is K=0.
TUM and ScanNet on an RTX 4090.
| Method | TUM ATE (cm) | ScanNet ATE (cm) | Depth AUSE |
| LoopSplat | 3.33 | 7.7 | n/a |
| VarSplat | 3.20 | 6.5 | 0.2752 |
| CG-SLAM | 4.0 | 8.1 | 0.1842 |
| BayesianGS | 3.12 | 6.7 | 0.1060 |
TUM average ATE is 3.12 cm, the lowest in the table. ScanNet is 6.7 cm, a hair behind VarSplat's 6.5, with tighter three-run scatter. Color AUSE 0.0121 matches VarSplat's 0.0123. Depth AUSE drops from CG-SLAM 0.1842 and VarSplat 0.2752 to 0.1060. Rendering is scored on training keyframes: TUM PSNR 23.30 vs LoopSplat 22.72 and VarSplat 23.14. That is not a novel-view contest.
Keyframes: fr1/desk 591 → 137, fr2/xyz 3393 → 744, fr3/office 2514 → 545, about 70% less. FPS is 0.56, faster than uncertainty-aware VarSplat (0.47), slower than LoopSplat without uncertainty (0.59).
Tracking-objective ablation: plain L1 dies on fr1/room; Gaussian NLL averages 42.44 cm ATE; the log-term variant 49.68 cm; normalized L1 averages 4.18 cm and tracks all five sequences. Enabling tracking plus keyframe modules moves TUM ATE from 3.47 to 3.12 and keyframes from 1689.8 to 380.6.
Uncertainty that only lives in the mapping loss barely helps SLAM. Here the same predictive variance down-weights bad pixels and skips frames the map already explains. The depth AUSE gap says opacity jitter is a geometric signal; color-only variance is not enough. Handheld and robot-mounted RGB-D maps are qualitative, but they at least leave the usual benchmarks.
Tracking is incremental: 0.08 cm better than VarSplat on TUM. The saving that matters is mapping calls.
Rendering is evaluated on training keyframes, not novel views. ScanNet ATE does not beat VarSplat. K=0 can insert on any positive surprise; the threshold curve is missing. Fewer mapping updates delay refinement when the criterion misses an informative frame. First-order linearization plus a Gaussian assumption can fail under heavy occlusion. FPS stays below 1, not onboard real-time. Robot tests are qualitative.