SFVO: Decoupled Confidence-Guided Stereo-Flow Visual Odometry with Bidirectional PnP
Kai Zhang, Guoyang Zhao, Jun Ma
cs.CV, cs.RO
2026-09-18
SFVO freezes UniMatch for stereo and flow, then runs bidirectional PnP with separate rotation and translation confidences. On an unseen campus UGV sequence, ATE is 4.91 m versus 12.28 m for stereo ORB-SLAM3.
Most learned visual odometry is still monocular, and monocular vision has a scale ambiguity unless an IMU or another scale cue is added. Stereo gives metric scale for free, but end-to-end stereo VO is heavy to model, so the literature is thinner.
Stereo matching and optical flow already produce dense correspondences. Disparity turns pixels into metric 3D points; flow ties those points to 2D observations in the next frame. That is a dense set of 3D–2D constraints. What remains is which points to trust, and how they should contribute separately to rotation and translation.
SFVO does not regress pose from pixels. Two rectified stereo pairs go through pretrained UniMatch for left–right disparity and forward/backward flow, kept at 1/8 resolution to cut memory. Disparity becomes metric depth from focal length and baseline.
The distinctive piece is decoupled confidence. In image motion, the translational term falls with depth while the rotational term does not; stereo depth error also grows with distance squared. Far points help rotation and hurt translation. A single scalar cannot carry both roles. The network therefore predicts rotation and translation channels in both temporal directions, four maps in total.
Pose is a differentiable bidirectional PnP that alternates rotation and translation. Rotation is a weighted Wahba problem solved by SVD on SO(3). Translation projects points onto the plane perpendicular to the bearing and solves a weighted linear system. Training starts from identity rotation and zero translation; test time can warm-start from the previous motion. Correspondence is supervised with SSIM on warped images, pose with L1 translation and angular rotation error, and confidence with a one-sided floor at 0.15 so weights do not collapse toward zero.
UniMatch weights initialize the correspondence nets. Training runs five epochs each on KITTI 00–08 and a split of EuRoC, four RTX 4090s, learning rate 1e-4.
Indoor and outdoor tests compare against monocular DPVO, monocular-inertial BotVIO, and stereo ORB-SLAM3 with loop closure off. Monocular methods receive scale alignment; stereo methods receive rigid alignment only.
| Sequence | Method | ATE (m) | RPEt (m) | RPEr (deg) |
| EuRoC MH02 | SFVO | 0.087 | 0.002 | 0.033 |
| EuRoC MH02 | ORB-SLAM3 | 0.037 | 0.024 | 0.548 |
| KITTI 09 | SFVO | 9.830 | 0.016 | 0.039 |
| KITTI 09 | ORB-SLAM3 | 2.077 | 0.017 | 0.044 |
| Unseen UGV | SFVO | 4.906 | 0.043 | 0.312 |
| Unseen UGV | ORB-SLAM3 | 12.282 | 0.090 | 0.784 |
SFVO wins seven of nine EuRoC metrics and the lowest translational and rotational RPE on all three sequences. On MH04, ATE is 0.217 versus 0.112 for ORB-SLAM3, with RPE still lower; the authors blame missing backend optimization. On KITTI 09 and 10, translational RPE is best, while ATE trails ORB-SLAM3’s local window.
The unseen set is a campus UGV with industrial stereo cameras and RTK-GNSS ground truth, different look and slower motion than KITTI, unused in training. ATE is about 60% lower than the baselines. Confidence ablation: a shared map yields 15.006 m ATE on KITTI 09; decoupling reaches 9.830 and peaks in five epochs, versus more than 40 for the coupled model. Far regions pick up rotation confidence, near regions translation confidence, and pedestrians or moving cars are down-weighted, without explicit depth labels.
At 480×640, peak GPU memory is about 930 MB, end-to-end runtime about 45 ms on a 4090 and 130 ms on a 4060 Ti. Each solver iteration is about 1.1 ms; training uses 20 iterations, inference about 7 with a warm start.
Treat a strong correspondence network as the geometric front end, and VO only has to learn whom to trust and how to solve pose. Training is lighter, and newer stereo or flow backbones are easier to swap in. Decoupled rotation/translation confidence is the reusable geometric prior: keep far points for rotation, near points for translation. A single scalar cannot do that.
This is still frame-to-frame. Against a SLAM system with keyframes and a local map, ATE will lose, and KITTI 09’s 9.83 versus 2.08 already shows it. The strength is relative motion and cross-platform transfer, not a drift-free long trajectory.
No backend. The MH04 pattern of low RPE and high ATE is a design consequence. KITTI tests only 09 and 10 after training on 00–08, a split shared with many VO papers, so the road is not fully unseen. The UGV result is a single sequence; the variance behind “about 60%” is unknown.
Correspondences stay at low resolution, which is enough for pose and discards high-resolution structure. Dynamic objects are suppressed by confidence, not by semantics. The code “will be released,” so reproduction waits. If the goal is loop-closed global accuracy, this paper supplies a front end, not a full SLAM stack.