F$^2$ SLAM: Turning Feed-Forward Geometry into Persistent Factors for SLAM
Zhisong Xu, Fan Zhu, Jiawei Qian, Ziyu Chen, Zhenjun Zhao, Javier Civera
cs.CV
2026-10-04
F²SLAM turns feed-forward geometry into persistent BA factors shared with local tracking. On uncalibrated Replica, mean ATE drops from 0.030 m to 0.002 m.
Online dense SLAM revisits poses and depth with bundle adjustment, but the measurements mostly come from a short time window. Weak overlap, occlusion, or a viewpoint change, and drift accumulates along the sequence.
Feed-forward 3D models such as DUSt3R, MASt3R, and VGGT read many frames in one pass and emit depth, point maps, and camera poses over a much wider context. MASt3R-SLAM and VGGT-SLAM typically reconstruct a local window or a submap, then align or fuse that geometry with the online estimate. The wider evidence never enters the measurement set that bundle adjustment actually optimizes, and a one-shot correction is not reweighted when newer local observations arrive.
F²SLAM writes feed-forward geometry onto a persistent factor graph as measurements. Local tracking and wide-context geometry then constrain the same poses, inverse depths, and optional intrinsics.
The optimized state is one set of keyframe poses, dense inverse-depth maps, and, when calibration is unknown, intrinsics (focal lengths and principal point). Known calibration keeps intrinsics fixed. Feed-forward pose and depth may initialize this state. They do not add a second geometric state.
Each directed edge stores a target-weight measurement: the pixel where a source pixel should land in the other keyframe, plus a per-pixel confidence on the reprojection residual. Bundle adjustment penalizes the gap between the projection under the current state and that stored coordinate. The stream label and the lock bit never enter the objective, so the solver cannot tell the streams apart.
The high-frequency stream is a pretrained DROID-Net. It inserts a keyframe when average image motion exceeds 40 pixels, links the new frame to three temporal neighbors and two nearby keyframes in both directions, and predicts a correction to the current reprojection from a correlation volume. That correction refreshes unlocked edges and keeps the graph connected.
The low-frequency stream adds no edges. Inference runs once accumulated image motion reaches 800, and the buffer is capped at 20 keyframes so slow motion cannot wait indefinitely. DINO features pass through a multi-view transformer. Flow3r predicts pose, depth, and a full source-to-target displacement with confidence in bundle-adjustment coordinates. Confidence is scaled by a fixed 0.25. Pose and depth are gauge-aligned into the current SLAM frame, which also supplies an initialization. They do not overwrite the displacement target.
A candidate replaces the stored measurement only when it agrees with the current geometry. Pixels with a valid projection and confidence above a fixed floor form the support. Those whose reprojection error is also within 3 pixels count as inliers. The acceptance score multiplies mean confidence on the support, the support ratio, and the inlier ratio, then divides by one plus the mean inlier error. A pair is accepted when the support ratio is at least 0.05 and the score clears its threshold. Accepted edges stay locked against high-frequency overwrites and are removed after 20 frontend updates. Rejected candidates leave the old measurement in place.
One dense bundle adjustment then runs two iterations per graph update: a robust reprojection term, inverse-depth smoothness, and, if uncalibrated, a pull of the intrinsics toward a prior. Every edge contributes exactly one residual. A low-frequency refresh replaces that residual's target.
Trajectory error is Sim(3)-aligned absolute trajectory error (ATE), RMSE in meters. Uncalibrated runs do not see ground-truth intrinsics. Timing is on one A100 (40 GB).
| Setting | Baseline | Baseline | F²SLAM |
| Replica, uncalibrated | VGGT-SLAM 2.0 | 0.030 | 0.002 |
| TUM, uncalibrated | SLAM-Former | 0.039 | 0.027 |
| TUM, calibrated | MASt3R-SLAM | 0.030 | 0.029 |
| 7-Scenes, uncalibrated | SLAM-Former | 0.041 | 0.051 |
| 7-Scenes, calibrated | DROID-SLAM | 0.050 | 0.053 |
| KITTI, uncalibrated | VGGT-SLAM++ | 64.94 | 30.18 |
| KITTI, calibrated | DROID-SLAM | 554.82 | 67.19 |
| KITTI, calibrated | LDSO | 22.43 | 67.19 |
On uncalibrated Replica every scene lands between 0.002 m and 0.003 m. Calibrated Replica ties DROID-SLAM at 0.003 m. Lighting is stable, blur is limited, and inter-frame motion is small, so the high-frequency tracker is already comfortable. Calibrated TUM beats MASt3R-SLAM by 0.001 m. On 7-Scenes, F²SLAM trails DROID-SLAM when calibrated and trails SLAM-Former when not.
Calibrated KITTI moves the average from DROID-SLAM's 554.82 m to 67.19 m. Sequence 01 supplies most of that gap: DROID-SLAM reports 5344.60 m there, F²SLAM 64.77 m. On the other sequences both systems are often still in the tens or low hundreds of meters, above loop-closing LDSO (22.43 m) and DPV-SLAM++ (25.75 m). Uncalibrated KITTI averages 30.18 m, ahead of VGGT-SLAM at 81.22 m. Sequences 08 and 09 remain at 75.98 m and 68.20 m.
Reconstruction skips confidence-based point filtering. On Replica, F²SLAM averages 0.0266 m accuracy and 0.0174 m completion, against 0.0270 m and 0.0187 m for SLAM-Former. DROID-SLAM's completion is 0.1229 m. On 7-Scenes, accuracy ties SLAM-Former at 0.024 m. Completion falls from 0.030 m to 0.015 m, and Chamfer distance from 0.027 m to 0.019 m, about 50% and 30% as stated in the paper.
The Replica ablation separates the pieces. Tracking alone already reaches 0.0027 m ATE, 0.0303 m accuracy, and 0.0188 m completion. The full model is 0.0023 m, 0.0266 m, and 0.0174 m. Removing the consistency gate sends ATE to 0.5848 m. Removing gauge alignment leaves ATE at 0.0024 m and completion at 0.0194 m. On the same GPU the system runs at 7.399 FPS and 15443 MB. SLAM-Former reaches 8.154 FPS at 27693 MB. VGGT-SLAM 2.0 runs at 3.909 FPS and 6523 MB. Windows of 4 or 8 keyframes fail. From 12 upward the curves flatten, and the default is 20.
The useful regime is an indoor sequence with poor or unknown intrinsics. Poses, inverse depth, and intrinsics move in one bundle adjustment, and uncalibrated TUM and Replica come out ahead of the feed-forward SLAM systems in the table. Completion moves more than calibrated trajectory error, because a measurement that survives the gate can fill regions local tracking barely sees.
On short calibrated indoor clips the gain over DROID-SLAM and MASt3R-SLAM is small, and on 7-Scenes the trajectory is slightly worse. Without loop closure, KITTI stays at tens of meters. The piece that carries the system is the gate. Turn it off and Replica ATE jumps from 0.0023 m to 0.5848 m. Turn it on and the feed-forward refresh only moves tracking-only ATE from 0.0027 m to 0.0023 m.
DROID-Net, DINO, and Flow3r are frozen. The backend sees only target coordinates and weights, so another feed-forward model can be attached if it emits displacement and confidence in the same coordinates. About 7.4 FPS and 15 GB on an A100 is an offline or near-online budget.
There is no explicit loop closure, so long outdoor trajectories keep drifting. Low-frequency quality follows the feed-forward model, which weakens under weak texture, dynamic objects, large viewpoint changes, and scale ambiguity.
The drop from 0.030 m to 0.002 m is against VGGT-SLAM 2.0. A control that keeps Flow3r and only changes submap alignment into factor refresh is not in the paper. Tracking alone is already at 0.0027 m in the ablation, and the refresh shaves off another 0.0004 m. Calibrated 7-Scenes moves the wrong way, from DROID-SLAM's 0.050 m to 0.053 m. No dynamic-scene split is reported. Skipping confidence filtering leaves the completion gain partly ambiguous, since some of it can be a denser cloud. Thresholds are shared across datasets. Geometric tolerances of 2, 3, and 4 pixels behave similarly in the appendix, and there is no curve for the case where the gate rejects most low-frequency candidates and the system falls back to tracking.