multi-view correspondence estimation

This paper addresses the significant performance degradation of conventional handcrafted methods in cross-modal feature matching—e.g., RGB/depth/point cloud/LiDAR/medical images/visual-language—caused by modality heterogeneity. To this end, we propose a modality-aware unified matching framework that jointly integrates geometric-aware descriptors, sparse-dense point cloud co-modeling, attention-enhanced networks, and cross-modal alignment mechanisms. The framework is architecture-agnostic, supporting both CNN- and Transformer-based backbones, and accommodates both detector-based (e.g., SuperPoint) and detector-free (e.g., LoFTR) paradigms. We systematically survey and empirically evaluate state-of-the-art single- and cross-modal matching approaches. Extensive experiments across diverse heterogeneous modality pairs demonstrate substantial improvements in robustness, generalization, and matching accuracy. Notably, detector-free deep models exhibit superior performance in cross-modal settings, highlighting their strong adaptability to modality shifts.

This paper addresses the relative pose estimation problem for three calibrated cameras given only four correspondences across all views. To overcome limitations of conventional methods—namely, their reliance on more correspondences or insufficient robustness—we propose a novel strategy that approximates a fifth correspondence using the centroid of the four observed points. We further introduce the first joint three-view pose estimation framework integrating a 4-point affine fundamental matrix solver, a standard 5-point relative pose solver, and a P3P solver. Geometric modeling enhances robustness against noise and outliers, while local optimization refines accuracy. Evaluated on real-world datasets, our method achieves state-of-the-art performance: the centroid-based strategy significantly outperforms pure affine approaches, striking a superior balance among accuracy, robustness, and computational efficiency, with straightforward implementation.

In large-scale Structure-from-Motion (SfM), sparse inter-view overlap and drastic viewpoint changes—especially in aerial-to-ground scenarios—lead to low cross-image feature matching density and weak geometric consistency. To address this, we propose a geometry-guided hybrid matching paradigm: (1) geometric verification is formulated as an optimization problem based on Sampson distance; (2) detector-agnostic dense matching is fused with detector-driven sparse anchor guidance, where sparse anchors constrain and enhance the geometric consistency of dense matches; and (3) multi-view geometric consistency is explicitly modeled. Our method significantly improves both matching density and accuracy, outperforming state-of-the-art approaches in extreme large-scale settings. Consequently, camera pose estimation becomes more accurate, and the reconstructed 3D point cloud achieves higher completeness and fidelity.

This study addresses the problem of recovering true correspondences from multiple noisy and independently permuted point clouds in high-dimensional space. While single-view observation exhibits an information-theoretic impossibility threshold—where exact matching becomes infeasible when the signal strength parameter $b < 2$—this work demonstrates for the first time that incorporating multiple views circumvents this fundamental limitation. Leveraging a high-dimensional Gaussian model and tools from random matrix theory, the authors devise a polynomial-time algorithm that, given $K$ views, achieves near-perfect recovery with only $o(n)$ mismatches whenever $b > K/(K-1)$. Notably, with three views, the method enables efficient and accurate matching in the regime $3/2 < b < 2$, which is provably impossible under a single view.

This work proposes a novel method for establishing point correspondences across image sequences in real time under unknown 3D scene structure and imaging geometry. The approach introduces a channel-vector-based uncertainty density model and employs an online optimization mechanism driven by Neyman chi-square divergence to iteratively learn mappings between image point sets. By representing channel vectors with basis functions and integrating a density divergence criterion, the algorithm achieves rapid convergence and high-accuracy correspondence estimation under general imaging geometries. Experimental results demonstrate that the proposed method outperforms state-of-the-art techniques across multiple metrics, offering a compelling combination of real-time performance, robustness, and accuracy.

This study addresses the insufficient robustness and accuracy of local feature matching in overlapping regions of satellite imagery. To this end, the authors construct a manually curated satellite image dataset annotated with GPS coordinates and conduct a systematic evaluation of SIFT and ORB algorithms across the entire matching pipeline—including keypoint detection, descriptor extraction, feature matching, and RANSAC-based geometric verification. Using the inlier ratio as the primary metric for matching quality, the work quantitatively analyzes the impact of keypoint quantity on matching performance. The results reveal a nonlinear relationship between the number of detected keypoints and the inlier ratio, offering empirical evidence and theoretical guidance for algorithm selection and parameter tuning in remote sensing image matching tasks.

This work addresses the failure of conventional SIFT-based image registration in scenes dominated by strong linear structures, where local features become ambiguous and poorly discriminative. To overcome this limitation, the authors propose a novel approach that, for the first time, transfers SIFT descriptors into Hough space for matching. By leveraging the Hough transform, linear structures are mapped to prominent peaks, thereby restoring the distinctiveness of features. The resulting Hough-space feature matching framework significantly outperforms standard SIFT in highly structured environments while maintaining comparable registration accuracy in general scenes. This method effectively mitigates the performance degradation of SIFT in structured settings, offering a robust solution for reliable image alignment across diverse scenarios.

This work addresses the challenge of jointly reconstructing dense dynamic scenes and estimating camera poses from multiple freely moving cameras, overcoming limitations of monocular inputs or reliance on pre-calibrated rigid camera arrays. The authors propose a two-stage optimization framework: the first stage constructs a spatiotemporal connectivity graph that extends visual SLAM to multi-camera settings by integrating temporal continuity and spatial overlap to achieve consistent scale and robust tracking; the second stage jointly optimizes dense depth and camera poses using wide-baseline optical flow. Key innovations include the spatiotemporal graph structure and a wide-baseline initialization strategy, which significantly enhance robustness in low-overlap scenarios. The study also introduces MultiCamRobolab, the first real-world multi-camera dataset with motion-capture ground truth. Experiments demonstrate superior performance over existing feedforward models on both synthetic and real data, with reduced memory consumption.

This work addresses the lack of explicit modeling of co-visible regions in image correspondence estimation under large viewpoint and scale variations by proposing a structured feature matching method grounded in co-visibility modeling. It extends the Segment Anything Model (SAM) to multi-view correspondence inference for the first time, leveraging predicted cross-view co-visible masks and bounding boxes as structured priors. A symmetric cross-view interaction mechanism is introduced to enable bidirectional feature exchange and semantic alignment. By integrating mask–box consistency constraints with a unified supervision strategy, the approach shifts the matching paradigm from pixel-level to region-level. The method achieves significant performance gains over existing techniques across multiple challenging benchmarks, demonstrating notably enhanced robustness under extreme viewpoint and scale changes.

This study addresses the trade-off in image–point cloud registration between insufficient inliers and an excessively high outlier ratio caused by suboptimal point cloud density, which limits registration accuracy. It presents the first systematic analysis of how point cloud density affects cross-modal registration and introduces a cross-coordinate correspondence pruning mechanism. Specifically, coarse correspondences are projected into the image coordinate system, where a lightweight network fuses geometric and feature information to predict inlier confidence scores for effective outlier rejection. Furthermore, a multi-density point cloud ensemble strategy is employed to enhance inlier recall. The proposed method consistently outperforms existing approaches across multiple benchmarks, achieving a registration recall improvement of at least 8.6%.

We present an efficient and robust method for 3D geometric reconstruction that is based solely on the camera-independent linear relationships among a given set of points, which are stable over time and robustly estimated using multiple point matches. We essentially learn, from correspondences between points across several frames, a linear geometric auto-regression matrix $\mathbf{W}$, which establishes how a point in 3D can be expressed as a linear combination of all the others. This matrix is constant and does not depend on the world coordinate system or the camera pose---it is an intrinsic property of the point set. We also show that the principal eigenvectors of $\mathbf{W}$, which all have eigenvalue $1$, provide a homogeneous representation of the 3D point configuration. The first version of our method takes advantage of noisy monocular depth maps in order to obtain, from multiple frames, a robust geometric auto-regression matrix $\mathbf{W}$ of linear relationships between the 3D points. Thus, we build on recent advances in deep learning, which now provide monocular depth estimation models that are fast but very often noisy. Our approach handles noise through robust linear estimation over several frames. The second version of our method does not need monocular depth estimation maps. It applies in cases of weak-perspective projection, when the linear combinations between the 3D points can be robustly estimated from their 2D projections in the image. Note that the camera projection matrix is never used in our derivations. Consequently, our method does not recover camera pose, but only 3D structure. This is a key difference between our method and the related literature on 3D geometric reconstruction.