Depth Anything V4: Dynamic 4D Scene Reconstruction via Riemannian Flow Matching on 4D Gaussian Splatting
Jiaming Fan, Jian Lu, Jinling Jia, Chenbin Zhang
cs.CV
2026-08-19
NJUPT applies Riemannian Flow Matching to 4D Gaussians and claims +0.044 F-score over a matched MLP. Withdrawn two days later for major research errors; key tables contradict.
The Depth Anything line turned monocular depth into a feed-forward default: V1 on unlabeled data, V2 with synthetic refinement, V3 with multi-view geometry. All three are discriminative regressions. One frame in, one point estimate out. No time axis, no volumetric field.
This Nanjing University of Posts and Telecommunications group branded a sequel Depth Anything V4 and switched the job. The task is dynamic 4D reconstruction from monocular video, stored as 4D Gaussian Splatting. Scale lives on the positive orthant, rotation on SO(3), opacity in (0, 1). Linear interpolation in the raw parameter space produces illegal Gaussians: negative scales, non-unit quaternions, which then need clipping or projection. The claimed fix is to put the probability path on the manifold so every intermediate state stays valid.
The PDF appeared on 18 August 2026. On 20 August the first author, Jiaming Fan, withdrew it with the comment "Major errors in research." What follows restates the claims, then checks the tables against each other.
A frozen DINOv2-L backbone feeds a Perceiver that compresses a variable set of Gaussians into 512 tokens. The prior is Riemannian Flow Matching: a geodesic probability path from noise to a target Gaussian field, with a vector field that also conditions on image features and the video time index τ. An ODE integrator then emits about 100,000 Gaussians.
Manifold ops are split by parameter:
The teacher is a 4DGS head. Its losses cover photometric rendering, SE(3) equivariance with rigidity scores, RAFT optical-flow consistency, and a scale regularizer that takes the median depth of Depth Anything V2. The text repeats that no human depth labels enter the training losses, and in the same section calls DA2 depth weak supervision.
Sampling is two-stage: 10 coarse ODE steps for layout, 20 fine steps for detail. The fine stage can add the rendering-loss gradient into the vector field, then run 20 steps of test-time optimization at learning rate \(10^{-3}\). A deterministic MLP baseline shares WebVid-10M + MegaDepth, the same architecture, and the same TTO protocol; only the objective changes, from flow matching to L2 regression. Pre-training is listed as 360 GPU-hours.
The headline table tries to split three factors on Dynamic-Sim (described as 100 synthetic scenes with 4DGS ground truth) and KITTI Dynamic (50 sequences):
| Method | Dynamic-Sim F1 | KITTI F1 |
| 4D-GS 100 iter | 0.702 | 0.552 |
| 4D-GS + TTO | 0.748 | 0.598 |
| MLP + TTO | 0.762 | 0.615 |
| RFM + TTO | 0.806 | 0.652 |
The decomposition is TTO +0.046, pre-training +0.014, and RFM +0.044 over the matched MLP. On KITTI the isolated RFM gain is written as +0.037. Fully optimized 4D-GS at 5000 iterations scores 0.781 / 0.623, below DAV4 with 20 TTO steps.
Under a shared 20-step TTO budget for novel-view synthesis, ETH3D PSNR is 25.13 dB for DAV4, 22.34 dB for randomly initialized 3D-GS, and 24.12 dB for 3D-GS at 5000 iterations. ScanNet is 23.82 / 21.58 / 23.05 dB. Uncertainty comes from variance across 10 ODE trajectories: NGLL -1.58 and ECE 0.058, against MC-Dropout at -0.82 / 0.18. Euclidean flow matching plus projection scores F1 0.684 with 12.3% invalid Gaussians; the Riemannian version is written as 0.806 and 0.01%. Per-scene inference is 420 ms, set against the 38 ms quoted for DA3.
Do not treat those figures as settled results.
For anyone tracking this area: do not use this paper as a reproducible baseline, and do not read it as an official Depth Anything sequel. The authors are from NJUPT's AI and automation colleges, not the Yang et al. group behind V1–V3. The task moved from monocular depth to offline 4D reconstruction. Comparing 420 ms against DA3's 38 ms mixes two product classes.
Putting flow matching on the 4DGS manifold is a coherent research idea. Scale and rotation of Gaussians are not Euclidean. This particular implementation, dataset, and comparison have already been pulled from the public record. Anyone pursuing the idea should go back to 4D-GS, Deformable-GS, and the original Riemannian flow-matching papers.
The authors list four limits of their own: 420 ms is too slow for robotics; 360 GPU-hours of pre-training is heavy; the 3D-GS novel-view comparison starts from random init while DAV4 starts from a learned prior; the rendering gradient is not a conservative field, so TTO guidance has no energy interpretation.
Harder problems sit in the tables.
The time-conditioning ablation is labeled KITTI Dynamic: F1 0.742 without τ, 0.806 with τ. In the main table, 0.806 is the Dynamic-Sim score for RFM+TTO; the same model on KITTI is 0.652. The KITTI ablation reuses the synthetic-set number.
The cost tables disagree as well. The main text writes \(420 + 360 \times 3600 / 1000 = 1716\) ms per scene at N=1000, adding GPU-seconds to milliseconds. The appendix amortizes in milliseconds and lists \(1.296 \times 10^5\) ms at N=10,000; the same cell in the main table is 550 ms. Two "corrected" cost tables cannot both be true.
"No human-annotated depth labels as training losses" and "DA2 median depth for scale" appear in the same paragraph. The bibliography assigns Neural Scene Flow Fields to Wang et al. 2021; the standard authorship is Li et al. Dynamic-Sim is cited as an ICCV 2023 dataset with 4DGS ground truth, while 4DGS itself was published in 2024, so the timeline does not work. Version 1 of the abstract already says "corrected computational cost analysis," which reads like a leftover from an earlier draft.
The 20 August withdrawal note is "Major errors in research." On the numbers that can be checked inside the PDF, the headline claim of an isolated +0.044 F-score from RFM does not survive a table-to-table read.