QuacamFM: Quaternion-Constrained Flow Matching for Camera Pose Estimation
Bao-Long Tran, Cuong Le, Tahereh Dehdarirad, Fredrik Viksten, Per-Erik Forssén
cs.CV
2026-09-26
QuacamFM maps noise to camera poses with slerp on unit quaternions. CO3Dv2, 10 views: 90.5 RRA@30 vs PoseDiffusion 86.7, 0.46s on A100, no correspondences.
Estimating camera pose from an unordered sparse set of views is underconstrained. Classical SfM needs correspondences and bundle adjustment; with little overlap the matches never form, and COLMAP often fails to converge (missing frustums in Figure 3). PoseDiffusion and related probabilistic models sample multiple pose hypotheses. They still store rotations as unit quaternions, then add and remove noise as if those were free 4D vectors. Once the unit-length constraint breaks, the 4-vector is no longer a 3D rotation, and the sampling path chatters on the sphere.
The representation lives on a manifold. The generator ignores that.
QuacamFM, from Linköping University, splits rotation and translation into two flows.
Rotation is flow matching on unit quaternions. The source is a random axis-angle converted to a unit quaternion; the target is the ground-truth rotation. The optimal-transport path is spherical linear interpolation (slerp): a geodesic on S³ with constant angular speed, the great-circle route that stays on the surface. An 8-layer Transformer (8 heads, FFN width 1024) reads the current quaternion, time t, and image features, and regresses angular velocity ω with an L1 loss against that geodesic velocity. Integration is multiplicative. ω is turned into an incremental quaternion and composed with the current pose by the Hamilton product, which is quaternion multiplication used to concatenate two rotations and does not commute. Every step stays on S³.
Translation uses ordinary Euclidean lerp flow matching, integrating Gaussian noise to the target translation.
A control variant, QuacamFM-lerp, treats the quaternion as a 4D vector, interpolates in a straight line, and renormalizes after each step. Same architecture, same hyperparameters, only the transport path changes. That isolates the constraint.
Visual conditioning is a fine-tunable DINO-ViT. Images are cropped around the object, resized to 224×224, and features at scales 1, 1/2, and 1/3 are averaged. One frame is picked at random as the reference, placed at the origin with identity rotation; translations are divided by the median of their transformed norms. Training samples 3 to 30 frames per step. The total loss sums angular-velocity, translation-velocity, reconstructed-rotation, and reconstructed-translation terms with weights 1.0, 1.0, 0.1, and 2.0.
CO3Dv2, 10 input frames, measured on an A100:
| Method | RTA@30 | RRA@30 | AUC@30 | Time (s) |
| COLMAP+SPSG | 28.2 | 44.4 | 25.1 | 3.26 |
| PoseReg | 61.3 | 65.5 | 45.5 | 0.03 |
| RelPose++ | 84.3 | 88.7 | 66.5 | 4.49 |
| PoseDiffusion | 79.9 | 86.7 | 66.5 | 8.83 |
| PoseDiffusion w/o GGS | 72.8 | 82.8 | 56.0 | 0.39 |
| QuacamFM-lerp | 77.2 | 75.8 | 51.3 | 0.46 |
| QuacamFM | 86.7 | 90.5 | 67.3 | 0.46 |
QuacamFM leads on every @30 metric. At the tighter @15 threshold, RTA@15 is 74.2, below PoseDiffusion with geometric guidance (79.8). RRA@15 is 82.0, below RelPose++ at 84.5. Both of those baselines pay for extra optimization. PoseDiffusion runs GGS (geometric guidance over correspondences) on the last 10 denoising steps, 8.83s for 10 frames. RelPose++ runs 50 rotation-consistency iterations, 4.49s. PoseDiffusion without GGS takes 0.39s, in the same band as QuacamFM's 0.46s, and QuacamFM wins that matched comparison across the board.
The lerp variant drops RRA@30 from 90.5 to 75.8, worse than PoseDiffusion without GGS (82.8). Renormalizing after the fact does not fix a Euclidean chord cutting through the sphere.
On 10 held-out CO3Dv2 categories, QuacamFM reaches 64.9 RRA@15 and 51.1 AUC@30, slightly above correspondence-using PoseDiffusion (62.8 / 50.8). RealEstate10K is zero-shot: rotation RRA@15 is 78.3, tying PixSfM and beating PoseDiffusion's 76.8; translation RTA@15 is only 50.6, behind PoseDiffusion at 67.2 and PixSfM at 64.2, and AUC@30 is 42.7 versus PixSfM's 49.4. Translation learned on turntable object videos does not transfer to free indoor and outdoor motion.
ViT-DINO is the best backbone (82.0 RRA@15) versus supervised ResNet50 at 77.9 and ResNet50-DINO at 78.4. Thirty integration steps work best; 20 steps give 79.5 RRA@15, 50 steps 81.0, 80 steps 79.8. More steps are not monotonically better.
The design takeaway for sparse unordered pose is narrow and usable: if rotation is stored as a unit quaternion, the generator has to integrate on S³. Treating it as an R⁴ vector, adding noise, and renormalizing is the wrong geometry. The change is the integrator. Runtime stays around 0.46s and needs no correspondences or post-optimization.
Object-centric captures with few views and weak overlap, the setting where SfM dies, are the fit. For scene-scale free motion where translation has to be accurate, PoseDiffusion with GGS or PixSfM is still safer. This is a correct manifold constraint on top of an existing probabilistic pose pipeline, not a replacement for geometry.
The paper states the gap: no epipolar constraints, because correspondence extraction and the extra optimization are expensive. That is exactly why translation lags on RealEstate10K and why PoseDiffusion wins at @15.
A few other holes. CO3Dv2 ground truth itself comes from COLMAP on 200 frames, so using COLMAP as the sparse baseline is a somewhat circular comparison. In-the-wild captures are qualitative only. Training data is turntable object video; after the jump to RealEstate10K, translation drops, even though the abstract still says the method generalizes well. Rotation holds. Translation does not. Quaternion double cover (q and −q are the same rotation) is not discussed. The 30-step sweet spot, with more steps getting worse, is also left unexplained.