Robust Multi-Model Fitting through Learning Neighbor Regions
Chang Nie, Guangming Wang, Zhe Liu, Hesheng Wang
cs.CV
2026-09-14
LNR filters random minimal sets with a network, then scores hypotheses from neighbor-region features, lifting NYU-VP AUC@5° from CONSAC's 50.61 to 57.54.
Multi-model fitting has to recover several lines, vanishing points, planes, or motions inside noisy data. Sequential RANSAC peels inliers and fits the next model, so each point belongs to one instance and overlap is under-counted. Later two-step methods grow a hypothesis pool and then pick a combination. Too many bad minimal sets waste the optimizer, and overlap is pushed into a complicated energy.
Learning hits a non-differentiable wall: random sampling and minimal solvers do not provide stable gradients. CONSAC and PARSAC hand sampling probabilities to a network, and PARSAC also needs the instance count in advance. The missing design is a learning pipeline that never differentiates through sampling or the solver.
LNR turns the two-step template into a coarse-to-fine network that learns on point features, not on hypothesis parameters. Input is an unordered set of N points in c dimensions. Fourier positional encoding and a shared MLP produce point features; max-pooling produces a global feature; the two are concatenated to 256 dimensions. Minimal sets are then drawn at random, as in RANSAC, two points for a line and so on. A coarse network reads each minimal set's features, emits a confidence, and only the top sets are sent through the minimal solver. Bad samples no longer consume solves.
The fine stage draws a neighbor region around each coarse hypothesis: points closer than R, with a task-specific distance (Euclidean for lines, directional for vanishing points, reprojection for planes, normalized epipolar for motion). Region features plus residuals to the hypothesis enter a network with a refinement head and a score head. The refinement head predicts parameter increments; the score head replaces raw inlier counts. At test time, hypothesis-level non-maximum suppression removes duplicates. The instance count is not an input.
Three losses train together. Coarse confidence and fine scores are binary cross-entropy against ground-truth similarity; refinement is Smooth L1. Fine-stage losses flow back into the coarse stage, so the coarse hypotheses are the ones the fine stage can actually digest.
Four tasks. On synthetic multi-line fitting with 1 to 10 lines and 40 to 60 percent outliers, [email protected]° at ten lines is 77.51 versus CONSAC 74.84 and Sequential RANSAC 48.55. Vanishing points on NYU-VP reach AUC@5° / @10° of 57.54 / 71.93, above the specialist Lin et al. at 55.87 / 69.53, CONSAC at 50.61 / 64.29, and PARSAC at 51.68 / 64.61. Trained only on NYU-VP, the same weights stay best on YUD and YUD+ without fine-tuning. Runtime is 10.31 Hz; a tiny variant reaches 46.96 Hz, nearer PARSAC's 84.13 Hz, but PARSAC needs the instance count.
Two-view plane segmentation on AdelaideRMF has 2.91% mean misclassification versus Barath et al. 3.12% and CONSAC 5.23%. Two-view motion is 5.75% for LNR versus 5.33% for Barath et al.; that task does not win, and the tiny variant is 6.27%. Ablation: removing the coarse stage drops NYU-VP AUC@5° to 49.26; replacing the fine stage with inlier counts drops it to 44.21.
| Task | Metric | LNR | Strongest baseline |
| NYU-VP vanishing points | AUC@5° | 57.54 | Lin et al. 55.87 |
| Multi-line, 10 lines | [email protected]° | 77.51 | CONSAC 74.84 |
| Plane segmentation | misclass. % | 2.91 | Barath 3.12 |
| Two-view motion | misclass. % | 5.75 | Barath 5.33 |
Multi-model fitting shows up in reconstruction, mixed reality, and driving when planes and motions have to be separated. Neighbor features can sit on several hypotheses at once, so overlap is not a mutually exclusive clustering cut. Sampling and solvers stay classical; a new task mainly swaps the minimal solver and the neighbor distance. Vanishing-point transfer without fine-tuning is the cleanest evidence that region scoring travels better than a specialist architecture.
Two-view motion does not take first place. The four tasks are not a clean sweep.
The discussion argues for interpretability and does not include a limitations section. Neighbor radius and distance remain task-specific hyperparameters. The coarse stage still draws a large random pool; efficiency comes from fewer solves, not fewer samples. Multi-line data is synthetic, and the noise model is not a real line detector. On motion, LNR trails Barath et al. by 0.42 points; hypothesis NMS may over-suppress on point-to-model assignment for fundamental matrices. PARSAC is faster when K is known, so LNR's speed claim depends on K being unknown.