Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models
Seokhyun Youn, Dahyeon Kye, Sung-Ho Bae, Jihyong Oh
cs.CV
2026-08-11
Self-Geometry uses 2D pixel matches as pseudo ground truth to add multi-view geometric consistency to six 3D vision foundation models, GT-free, in under two minutes per scene.
Vision foundation models (VFMs) like VGGT, pi-3, and Depth Anything 3 predict depth, camera pose, and a pointmap (per-pixel 3D coordinates) in a single forward pass, with solid generalization. But they are pretrained without enforcing multi-view geometric consistency, because explicit constraints like bundle adjustment are too costly. So their predictions carry geometric inconsistency, worse where the model is already inaccurate.
Prior test-time adaptation (TTA) methods use an implicit self-consistency signal squeezed from the model's own pointmaps or features, which improves things only indirectly. The representative method, Free-Geometry, barely moves pose or depth and marginally helps the pointmap.
Self-Geometry (CMLab, Chung-Ang University) rests on one observation: 2D pixel correspondences from an external feature matcher, LightGlue, are themselves explicit multi-view geometric supervision and can serve as pseudo ground truth, with no labels needed.
Three components:
Across six VFMs (VGGT, pi-3, DA3-Giant/Large/Base/Small) and four datasets (7Scenes, ETH3D, ScanNet++, HiRoom), pose and geometry improve consistently, and Self-Geometry delivers the largest mean gains among the six.
| Setting | Metric | Self-Geometry |
| Wide-baseline outdoor (VGGT, ETH3D) | pose AUC@3 | +37.3% |
| Indoor fine structure (DA3-Small, HiRoom) | geometry F1 (w/o pose) | +70.4% |
The two baselines do far worse: Free-Geometry barely moves, and TCO outright collapses on pi-3 (mean pose AUC@3 down 91.5%). The pattern is that weaker base models gain more, with DA3-Small/Base jumping 70% to 85% on HiRoom geometry F1, so the method is most useful on under-fit models. Each scene takes under two minutes on a single RTX PRO 6000, far cheaper than classical bundle adjustment.
For anyone building 3D reconstruction on VFMs, this is a plug-and-play boost: no labels, no teacher distillation, no architecture lock-in, two minutes a scene at inference to patch in geometric consistency. On already-strong models (e.g. VGGT on ScanNet++) the gain is small or occasionally negative on a few columns, so its value is concentrated on weak models and hard scenes (wide baselines, low texture).
The authors list two: supervision quality is bounded by LightGlue's matching, which fails on repetitive or textureless surfaces and wide baselines with little overlap; and per-scene latency of minutes is nowhere near real time. A further caveat: the small gains on strong models suggest this is a crutch for weak models more than a universal amplifier. The 42.4% gradient-conflict rate also hints that with a different loss mix the disentanglement logic may need retuning.