AdaptVPR: Route-Aware Hard Positive Generation for Robust Visual Place Recognition
Shunpeng Chen, Jingyi Zhang, Changwei Wang, Shengpeng Xu, Yukun Song, Xingtian Pei, Jinzhou Lin, Li Guo, Shibiao Xu
cs.CV
2026-09-04
AdaptVPR routes weather, occlusion, and dual edits with geometry checks to build 160K same-place hard positives, lifting occlusion R@1 by up to 9.2%.
Visual Place Recognition (VPR) treats localization as image retrieval: a query should return a database photo of the same or nearby place. Day versus night, rain versus sun, a bus in the foreground versus an empty curb can change appearance more than switching to a different street. If training mostly sees clean daytime street view, winter, night, and occlusion queries lose recall.
Collecting real revisits that cover weather, lighting, season, and traffic does not scale. Color jitter, crops, blur, and random erasing never produce a rainy night with wet-road reflections or a structured vehicle mask. Off-the-shelf diffusion fails in a different way. A convincing rain edit can rewrite facades, lane markings, or intersection topology. Treat that image as a positive in metric learning and the embedding is pulled toward a fake place. The useful object is a same-place hard positive: appearance hard enough to train on, geometry still of the original location.
A team at Beijing University of Posts and Telecommunications and Qilu University of Technology splits generation into three routes rather than one universal editor. AdaptVPR plans, routes, generates, then gates.
A Qwen3-VL-4B-Instruct pass writes a structured capability dict per reference image: weather-edit score, occlusion-edit score, and a bad-image flag for facade close-ups, signs, or frames with no road. A rule-based scheduler opens a route only if the matching score clears its threshold. When several routes qualify, a quota-plus-capability score picks one so samples do not pile onto easy edits. The four outcomes are Global Appearance, Local Occlusion, Dual, and Skip.
Backends differ by route. Global Appearance uses IC-Light to change weather, lighting, and time of day, one shot, reject on fail. Local Occlusion and Dual use Qwen-LightX2V to insert vehicles or pedestrians on plausible ground; Dual stacks a weather or night change on top. Dense traffic scenes can also drop a few cars and inpaint. Failed Local and Dual candidates get at most three prompt rewrites; the route and generator stay fixed.
Two gates decide admission. Geometric consistency runs SuperPoint, LightGlue, and a RANSAC homography, then takes the inlier ratio. Appearance diversity is CLIP feature distance. Both must pass. Thresholds are route-specific: geometry 0.78 / 0.82 / 0.72 for Global / Local / Dual, diversity 0.15 / 0.09 / 0.20, relaxed to 0.12 for rain-plus-vehicle Dual because rain and a foreground car already kill correspondences.
From 88,989 GSV-Cities references the pipeline keeps AdaptCities: 160K verified synthetic hard positives, 50,239 Global (31.4%), 45,804 Local (28.6%), 63,957 Dual (40.0%). Training concatenates them with the original set. Backbone, aggregator, and loss stay untouched. An 8:1 real-to-synthetic batch mix works best.
Plugging the set into SALAD, BoQ, EDTformer, and ImAge, with no architecture changes, raises R@1 on all 20 standard pairs (four models, five datasets). Pitts30k moves only 0.1 to 0.7 points; that urban-viewpoint set is already saturated. On seasonal Nordland, BoQ goes from 90.6% to 95.9% (+5.3). On SF-XL-v1, SALAD goes from 88.6% to 93.0% (+4.4).
Harder shifts are where the data pays.
| Setting | Baseline | +AdaptVPR | Δ R@1 |
| BoQ / SF-XL-Occlusion | 43.4 | 52.6 | +9.2 |
| BoQ / Nordland | 81.3 | 89.6 | +8.3 |
| EDTformer / Nordland | 73.1 | 81.2 | +8.1 |
| EDTformer / SF-XL-Occlusion | 51.3 | 57.9 | +6.6 |
| ImAge / SF-XL-Occlusion | 56.6 | 63.2 | +6.6 |
| SALAD / Nordland | 76.0 | 82.0 | +6.0 |
SF-XL-Night gains sit between 1.7 and 3.2 points. Pitts250k barely moves; BoQ drops 0.5, the only R@1 regression in the paper. SALAD is flat on SF-XL-Occlusion.
Ablations argue against dumping every synthetic frame. Geometry or diversity alone is unstable; using both cuts the pool to 65.6% retained and wins. Local Occlusion by itself lifts SF-XL-Occlusion R@1 from 50.0% to 52.6%; all three routes together win overall. Versus random one-shot generation, the full loop raises accept rate from 50.8% to 65.6% at 2.24 attempts and 46.5 s per image. SALAD improves on both DINOv2-B and DINOv3-B. Mixing denser than 8:1 (4:1, 2:1, 1:1) hurts; 2:1 and 1:1 fall slightly below real-only on SF-XL-v1.
This is a training-set patch. Inference adds no extra module, descriptor width, or index storage. If a VPR training loop already exists, the cheap path is to merge AdaptCities and sample 8:1. Gains concentrate on season, night, and occlusion, the slice original GSV-style data covers worst. Do not expect a leaderboard jump on sunny Pittsburgh streets.
Generation is not cheap: 2.24 tries and 46.5 s per Local or Dual image, all offline. Code and data are on GitHub.
The authors flag three issues. Homography inliers are a proxy, not a proof the synthetic image is still the same place. Routes and thresholds are frozen, so unseen environments may not fit. Cost sits in offline generation.
A few gaps stay open. SF-XL-Occlusion has 76 queries; the headline +9.2 for BoQ sits on a tiny set. SALAD does not move there at all. A homography on street view, which is 3D plus moving objects, can look clean while a facade still drifted. Thresholds are said not to be tuned on test sets, yet ablations report those same tests, so some leakage is possible. GSV-Cities is Google Street View; redistributing 160K derivatives still depends on the source license and privacy rules.