Failure or Drift? Evaluating Monocular SLAM under Synthetic and Real-World Corruptions
Abhay Skaria Thomas, Shashank Agnihotri, Margret Keuper
ECCV 2026
cs.CV, cs.RO
2026-08-31
ORB-SLAM2 often drops tracking; DPVO and DROID-SLAM stay up but drift past 100 m. Image-space tests pick DPVO, rain/fog pick DROID-SLAM, real evening reverses it.
Monocular SLAM is still scored on clean trajectories. Field failures come from rain, fog, low light, blur, and compression. Real adverse datasets are faithful, but weather, route, camera, and motion change together, so the cause of a break is hard to isolate. Synthetic corruptions freeze the route and ground truth and edit only the image. That control is useful only if the engineering conclusion matches the real condition the proxy is meant to stand in for. This paper tests that external validity.
Failure mode cannot collapse into one trajectory error. A feature system can drop tracking and return nothing. A learned tracker can keep emitting poses while error accumulates. Empty output is easy to catch. A live but wrong pose can pass into mapping, planning, and control.
Three systems run without corruption-specific adaptation: ORB-SLAM2 for sparse features with loop closure, DPVO for learned patch odometry, DROID-SLAM for dense correspondence with global bundle adjustment. Clean baseline is KITTI odometry 00-10 in monocular mode. The controlled sweep uses 115 variants of sequence 00: 13 image-space transforms, 7 geometry-aware effects, 3 compounds, five severities each. Real reference is four 4Seasons neighborhood sequences matched at condition level to rain-plus-contrast, fog-plus-contrast, and brightness-plus-contrast.
Evaluation first records whether a run returns a trajectory that can be associated with ground truth, then reports Sim(3)-aligned APE and RPE only on valid runs. Missing output is never scored as low error. Learned-tracker comparisons use only conditions where both results exist.
All three systems return an evaluable trajectory on every clean KITTI sequence. ORB-SLAM2 has the lowest median conditional APE at 8.2 m, then 518.0 m on sequence 01. DPVO and DROID-SLAM sit at 64.9 m and 82.0 m median; which of the two is better depends on the route.
Corruption flips the failure pattern. DPVO and DROID-SLAM stay valid on every archived synthetic attempt. ORB-SLAM2 is valid in 37 of 63 image-space runs, 15 of 23 geometry-aware runs, and 10 of 15 compound runs. Gaussian noise, impulse noise, and snow yield no evaluable ORB trajectory at any recorded severity. The learned systems pay for continuity: aligned APE often exceeds 100 m under geometry-aware and compound effects.
Which learned tracker looks robust depends on the corruption model. DPVO has lower APE in 30 of 43 paired image-space settings, including every brightness and contrast severity. DROID-SLAM wins all 33 geometry-aware settings and 14 of 15 compound settings; the only reversal is fog-plus-contrast at severity 5. Under geometry-aware rain, DROID-SLAM APE falls from 111.9 m at severity 1 to 87.9 m at severity 5, while DPVO stays between 125.5 m and 129.2 m. Severity orders the corruption parameters, not estimation difficulty.
On real data, structured proxies keep the ranking: DROID-SLAM is better by 17.96 m on rain-plus-contrast and 16.40 m on real rain, 2.27 m on fog-plus-contrast and 5.15 m on real winter. Evening reverses it. Brightness-plus-contrast still favors DROID-SLAM by 5.51 m; the real cloudy evening favors DPVO by 10.85 m. A Spearman ρ of 1.00 between two synthetic illumination curves still fails to preserve the real ordering.
A live pose and a trustworthy pose are different operational events. Learned trackers replace catastrophic loss with sustained drift. If a stack only checks whether output exists, a badly drifted estimate looks healthy. Protocols should report validity first, then error conditional on a valid trajectory.
The ranking warning is concrete. An ImageNet-C-style global-transform leaderboard would pick DPVO. Depth-aware rain and fog pick DROID-SLAM, and that ranking survives real rain and winter. A brightness proxy does not. Synthetic corruptions are diagnostic interventions, not calibrated forecasts of field error.
The controlled sweep uses one KITTI trajectory and one archived run per condition, so scene dependence, initialization variance, and runtime noise are unmeasured. Validity is binary; it does not say how much of a truncated trajectory is supported in time. 4Seasons matches conditions, not the same route and camera, so absolute errors are not comparable. The study is monocular and trajectory-centered: no inertial fusion, map accuracy, completeness, runtime, or memory. The paper also does not ablate dense correspondence, recurrent updates, and global optimization, so architectural explanations stay consistent with the design rather than causal.