3DGS Meets Factor Graph SLAM: Unifying Pose Optimization and Rendering in GTSAM

fdellaert · x · 2026-08-06

Current 3D Gaussian Splatting (3DGS) SLAM systems typically optimize camera poses via gradient descent through a differentiable rasterizer. Even when loop closure is applied, it often remains decoupled from the rendering objective, making global consistency difficult.

The author proposes expressing the 3DGS rendering process directly as a factor within GTSAM. By computing photometric residuals between rendered images and keyframes, and providing the corresponding Jacobians, this approach unifies pose optimization, loop closure, and multi-sensor fusion natively within the iSAM2 Bayes tree framework.

Original post →

More from Research

Research channel →