AutoCompass: Accurate Visual Localization on Public Maps by Learning from Weak Labels
Javier Tirado-Garín, Alan Savio Paul, Shuai Chen, Axel Barroso-Laguna, Tommaso Cavallari, Daniyar Turmukhambetov, Victor Adrian Prisacariu, Eric Brachmann
ECCV 2026
cs.CV
2026-09-03
AutoCompass trains neural map matchers from raw GPS and SfM relative poses, with no heading labels. KITTI DINOv2 plus relatives hits 70.8% lateral R@1 vs 48.7% for OrienterNet.
Neural map matchers correlate a bird's-eye feature from a ground image with a 2D map such as OpenStreetMap, and output planar position plus heading. Maps are free and tiny, about 200 KB for 128×128 m, unlike city-scale 3D models. Training still wants huge sets of images with absolute poses. Those labels usually come from SfM clusters fused with GPS. Systematic bias survives, and a sidewalk capture can be tagged inside a building. Strong supervision copies the error.
The backbone stays OrienterNet: image BEV features, map features, softmax over a discrete pose volume of size S×S×N (128 m tiles by default). Only the losses change.
Heading labels are dropped. Marginalize heading at the ground-truth position and maximize that cell's location probability. The geometric bias of BEV-to-map matching grows a heading on its own, from a single perspective image, without panoramas.
GPS is a weak location. Marginalize position and heading inside a ±r neighborhood of the GPS fix and maximize that chunk's mass. r in 5–20 m is insensitive and covers typical consumer GPS error. The network may put mass anywhere in the neighborhood; the matcher bias picks the point.
When relative poses exist, two more terms appear. Each image predicts an absolute pose distribution. Relative rotation is a circular cross-correlation of heading marginals; relative translation is a 2D cross-correlation of position marginals; NLL is taken at the SfM relative. Shared translational or rotational offsets cancel, which is exactly the failure mode of biased pseudo-GT. If the relative translation is not yet in the map frame, supervise the distance norm with a 5 m half-bin. Training samples pairs from the same SfM cluster with baseline under 100 m.
Training uses the 11 Mapillary Geo-Localization cities still available. KITTI Test2 has RTK ground truth.
| Method (OSM, 11 cities) | Lateral R@1 | Long. R@1 | Heading R@1 |
| OrienterNet ResNet-101 | 37.7 | 17.6 | 15.7 |
| AutoCompass raw GPS | 43.1 | 26.8 | 21.1 |
| AutoCompass relative poses | 56.6 | 33.2 | 28.9 |
| same + DINOv2 | 70.8 | 34.1 | 37.9 |
Raw GPS and no heading already beat OrienterNet retrained on optimized 3-DoF labels. Relative poses widen the gap and pass OSMLoc trained on 12 cities.
The same pattern holds on egocentric LaMAria (Zurich, cm-level control points) and Oxford Day-and-Night. LaMAria, single frame, DINOv2, relative poses: position R@1 20.9% versus 9.1% for OrienterNet on the same backbone; after 50-frame fusion, 86.2% versus 57.5%, and it beats GPS at tight thresholds. GPS is biased, so extra frames do not help it. Feature maps from strong supervision smear over whole buildings; weak supervision pins corners.
At city scale the bottleneck is labels, not the net. Heading can emerge from the matcher, GPS should be a neighborhood, and relative poses are cleaner than "optimized" absolutes. The losses are architecture-agnostic and drop onto OrienterNet or DINOv2. For urban AR or vehicle localization with consumer GPS plus a SLAM stretch, this supervision is cheaper than polishing pseudo-GT. Single-frame recall at coarse thresholds still sits near GPS; sequential fusion is where the method pulls away.
A coarse GPS fix is still required to crop the map tile. The default relative-translation loss wants geo-referenced tile origins; an arbitrary frame falls back to a weaker distance loss. Heading emergence is tied to BEV-map cross-correlation and may vanish under a regression head. Oxford geo-referencing itself has meter-level error, so evaluation aligns per sequence. LaMAria drops survey frames that look at the ground, a manual protocol choice. MGL is missing Amsterdam, so comparisons to 12-city checkpoints are imperfect; the paper retrains an 11-city OrienterNet for that reason.