Scalix: Uncertainty-Aware Scale-Consistent Monocular SLAM
Sebastian Barbas Laina, Tianyi Zhang, Panagiotis Petropoulakis, Simon Schaefer, Simon Boche, Jaehyung Jung, Cedric Le Gentil, Stefan Leutenegger
cs.RO
2026-08-18
Scalix folds a monocular depth model's per-frame scale into a factor-graph SLAM backend as an optimizable variable, cutting KITTI trajectory error 37% versus the next-best method.
A single camera cannot recover real-world scale. Feed a monocular SLAM (Simultaneous Localization And Mapping) system a video and it can reconstruct a trajectory and map that are geometrically correct but arbitrarily sized, since the whole reconstruction can be multiplied by any constant and still fit the images. Visual-inertial systems fix this by adding an IMU, but IMUs stop giving usable scale information when a robot moves at roughly constant velocity, which happens a lot in mobile robotics. Monocular depth foundation models such as Metric3Dv2 promise a shortcut: predict metric depth straight from a single image. The catch is that these models are noisy and their per-frame scale drifts, so plugging their raw output into a SLAM pipeline just trades one error source for another. Scalix asks how to use that imperfect depth signal without letting its noise poison the trajectory estimate.
Scalix splits a depth prediction into two pieces: metric depth equals a scale factor times an unscaled depth map. The scale factor is one number per frame, modeled as a Gaussian with mean 1 and a learned variance. The unscaled depth is per-pixel, also Gaussian, with its own learned per-pixel variance. Separating "how far off is this specific pixel" from "how far off is the whole frame's scale" is the central trick.
To produce both uncertainties, the authors freeze a pretrained Metric3Dv2 backbone and attach two small heads: one that pools global image features down to a single scale-uncertainty scalar, and one that outputs per-pixel depth uncertainty using the same architecture as the original depth head. Training only has ground-truth metric depth to supervise against, so both uncertainty sources get propagated back into a combined metric-depth covariance matrix. That matrix is not diagonal, because every pixel in a frame shares the same scale factor and is therefore correlated with every other pixel. Computing the resulting negative log-likelihood loss directly would mean inverting a dense matrix per image; the authors instead apply the Sherman-Morrison formula to collapse it into a closed-form, per-pixel summation.
With both uncertainties in hand, Scalix treats each frame's scale as a state variable inside OKVIS2's classical factor-graph backend (a probabilistic optimization framework where poses, landmarks, and constraints are nodes and edges). Every keyframe carries an extra scale state alongside its pose. Three residual terms drive the optimization: a standard reprojection error on 2D keypoints, a depth error comparing a triangulated 3D point's depth against the network's predicted unscaled depth (scaled by the current estimate), and a scale-prior error that pulls each frame's scale toward the network's prediction of 1.0, weighted by the learned uncertainty. To stay real-time, old keyframes get marginalized into a pose graph, and Scalix adds a relative-scale term to those graph edges so scale information survives marginalization instead of getting thrown away with the landmarks.
On the outdoor KITTI benchmark, Scalix (without full bundle adjustment) hits an average Absolute Trajectory Error (ATE, the RMSE between the estimated and ground-truth trajectory) of 4.60 meters under Sim(3) alignment, which ignores scale, beating the second-best method CUT3R's 7.31 meters (averaged over non-failed sequences) by 37%. Under SE(3) alignment, which does hold the system to producing correct metric scale, Scalix reaches 7.19 meters versus 12.07 meters for DROID-SLAM combined with the same Metric3D depth model, a 40% improvement. Adding full bundle adjustment pushes those numbers down further to 3.87 and 7.22 meters. Indoors on 7-Scenes, Scalix gets 8.6 cm ATE under Sim(3), trailing MASt3R-SLAM's 4.7 cm, but MASt3R-SLAM loses tracking entirely on all eleven KITTI sequences. Scalix is the only method in the comparison that holds up in both settings. On timing, keypoint matching takes 20 ms, depth-network inference 79 ms (only triggered on a subset of frames), landmark initialization 5 ms, and backend optimization 59 ms, with frontend and backend running on parallel threads on an Intel i7-13700 CPU paired with an RTX 3080 GPU. An ablation makes the case for scale optimization directly: removing it, KITTI sequence 01's ATE degrades from 10.72 m to 96.91 m under Sim(3) and to 575.58 m under SE(3), using the identical uncertainty model but without letting scale participate in the optimization.
Scalix does not train a new depth foundation model. It takes an existing one, Metric3Dv2, and turns the industry-wide complaint that "foundation model outputs are unreliable" into an explicit, learned-weight variable inside a probabilistic optimizer. That pattern generalizes past SLAM to any pipeline that fuses foundation-model priors across multiple views or frames. The system also runs on CPU, invoking the depth network on only a subset of keyframes rather than every frame, which matters for drones and other robots without a GPU to spare for online inference.
The authors attribute the indoor accuracy gap directly to Metric3Dv2's own weaker performance on indoor scenes, which means Scalix's ceiling is capped by whichever depth model it wraps. Scale is currently modeled as a single Gaussian hypothesis per frame; the paper flags multi-scale hypotheses as future work, meaning scenes with genuinely ambiguous depth, such as heavy occlusion or large textureless regions, are not explicitly handled yet. The uncertainty heads are trained on only two datasets, ScanNet and Waymo, and the paper does not test generalization to other robot platforms or sensor noise profiles. The conclusion also cites a 48% improvement in outdoor accuracy, a figure that does not obviously reconcile with the 37% (Sim3) and 40% (SE3) numbers reported in the results section, and the paper does not explain how that number was derived.