LBDU-VIO: Learned Bias Dynamics and Uncertainty for Visual-Inertial Odometry with Unreliable Vision
Qizhi Guo, Junning Lyu, Defu Lin, Shaoming He
cs.RO
2026-09-30
A neural ODE replaces MSCKF's random-walk IMU bias model, plus per-sample learned noise; EuRoC 10 s outages see position error 25.1% below the best baseline.
Aerial VIO runs on two clocks: cameras correct accumulated drift at roughly 20 Hz, while the IMU propagates state at 200 Hz in between. When vision degrades under motion blur, low texture, or a brief camera outage, the estimator leans on the IMU, and any modeling error there accumulates into drift. The root cause sits in two simplifications conventional MSCKF makes about the IMU: gyroscope and accelerometer biases evolve as random walks, and measurement noise is fixed after offline calibration. Low-cost MEMS IMUs do not behave that way. Their bias fluctuates with motion excitation and their noise varies with vibration, so the model mismatch eats accuracy directly.
Most learning-based fixes place the network outside the IMU integration. Denoising methods preprocess the raw stream and blur systematic bias together with stochastic noise in one window-level correction. Bias predictors that run at keyframe rate revert to the random walk between updates. In neither case does the learned quantity enter the per-sample integration.
LBDU-VIO puts two learned modules inside the MSCKF propagation step, so they shape both the predicted mean and the covariance.
The bias dynamics are learned with a neural ODE. Bias is the ODE state; the right-hand side is a small network (separate residual MLPs for gyroscope and accelerometer, hidden width at most 512) taking the current bias plus the interpolated IMU signal and its derivative. Because IMU samples are discrete, a causal cubic Hermite spline interpolates them so the dynamics can be evaluated between samples. In practice the integration is explicit Euler at the IMU rate (5 ms steps), producing one bias estimate per sample that is subtracted before navigation propagation. This replaces the random walk inside the filter, and the corresponding Jacobian blocks of the error-state Kalman filter are computed from the learned vector field.
The uncertainty model handles noise. A temporal encoder (CNN plus two GRU layers) reads a local window of raw IMU, and two MLP decoders output three-axis noise variances per sensor, mapped through exp to stay positive and assembled into diagonal covariances. These replace the fixed datasheet noise parameters in covariance propagation at every sample.
Training runs in three stages with pose supervision only, no ground-truth bias labels: first the gyroscope bias dynamics from orientation error, then the accelerometer dynamics from velocity and position error with the first model frozen, then the uncertainty model with both frozen, using a negative log-likelihood loss backpropagated through differentiable preintegration and the covariance recursion. A curvature regularizer penalizes differences between consecutive bias derivatives to keep the learned dynamics from wobbling.
The design choice that matters is placement. Both learned quantities participate in the state update at every IMU sample, so the mean and covariance are already shaped by the learned models before any visual update arrives, rather than sitting behind a preprocessing layer or a keyframe-level factor.
Under nominal conditions (vision continuously available), monocular LBDU-VIO ranks first on mean APE and AOE on both datasets:
| Method | EuRoC APE (m) | EuRoC AOE (deg) | TUM-VI rooms APE (m) |
| S-MSCKF (stereo) | 0.199 | 3.077 | 0.127 |
| MSCEqF | 0.329 | 3.275 | 0.235 |
| Brossard | 0.163 | 1.888 | 0.079 |
| VIO-IPNet | 0.154 | 1.760 | 0.229 |
| LBDU-VIO | 0.133 | 1.690 | 0.060 |
Mean APE is 13.6% below VIO-IPNet, the strongest baseline on EuRoC, 24.1% below the best monocular baseline on TUM-VI, and 52.8% below the stereo S-MSCKF there.
The outage experiments carry the headline claim. Deleting camera data for 1, 2, 3, 5, or 10 s starting at second 50 of each EuRoC test sequence (IMU untouched), LBDU-VIO posts the lowest mean RPE and ROE at every duration. RPE sits 13.2% to 26.7% below the best baseline for each duration, at 0.140 m versus 0.187 m in the 10 s case (25.1% lower). The ROE margin narrows with duration, down to 4.4% at 10 s. On MH 04 with a 10 s outage, trajectory APE is 2.74 m against 3.55 m for Brossard and 7.61 m for VIO-IPNet; the 1 s recovery RPE over the first 10 s after images return is 0.413 m, 46.7% below Brossard. On TUM-VI room4 with the same outage, mean APE is 0.1349 m versus 0.1920 m for the strongest baseline, 29.8% lower. Under synthetic visual distortion (a horizontal pixel shift ramped to 10 px, about 1.25 degrees), median APE RMSE is 0.101 m, 33.3% below the best baseline.
The ablation under a 3 s outage separates the modules. Raw IMU with fixed noise gives 0.249 m mean APE; learned bias alone cuts it to 0.158 m (36.5%), and adding the uncertainty model brings it to 0.111 m (a further 29.7%). The uncertainty model alone on raw IMU barely moves the result (0.249 to 0.240 m). Mean correction and covariance modeling only pay off together, consistent with what AirIMU reported.
This is a clean template for putting learned inertial models inside a filter. The MSCKF visual update is untouched; only the bias dynamics and noise parameters within inertial propagation are replaced, and training needs pose labels alone, no auxiliary sensors for bias ground truth. Anyone building a filter-based estimator can lift the recipe. The gain profile is clear as well: modest under nominal conditions, concentrated exactly where VIO fails in practice, with a monocular system beating a stereo baseline. For drone platforms tight on sensor and compute budgets, that trade is attractive. An incremental improvement, but each piece is backed by an ablation.
The authors concede that long inertial-only propagation remains hard, with the orientation margin down to 4.4% at 10 s, and leave longer outages and less structured degradation to future work. What the paper does not test: training and testing are split within the same datasets (6/5 on EuRoC, 3/3 on TUM-VI rooms), with no cross-dataset or cross-IMU evaluation, so portability to a new sensor is unverified. The TUM-VI evaluation covers only three indoor room sequences. The distortion test is a synthetic pixel shift, not real blur or lighting change. No online runtime is reported, so onboard real-time performance is inferred only from the small networks. Sequence-level wins are not uniform: stereo S-MSCKF keeps the lowest APE on MH 02, and on V1 03 the proposed AOE sits 49.8% above the MSCEqF minimum. The TUM-VI prose also swaps the names of the Brossard and VIO-IPNet baselines (the quoted percentages match the other column), which does not change the ranking but is worth checking against the table when citing numbers.