multi-view geometry

This paper addresses the fragmented and unsystematic modeling of geometric constraints in deep learning by proposing the first unified taxonomy of geometric constraints tailored for modern deep learning frameworks. Methodologically, it systematically integrates multi-view geometry, epipolar constraints, camera calibration models, self-supervised geometric consistency losses, and differentiable rendering to establish a three-dimensional classification framework spanning modeling principles, integration strategies, and optimization objectives. The contributions are threefold: (1) clarifying the applicability boundaries and failure mechanisms of over one hundred geometric constraints across vision tasks such as depth estimation; (2) uncovering key design paradigms for synergistic co-design of geometric priors and neural architectures; and (3) identifying principled pathways to overcome three core challenges—dynamic scenes, textureless regions, and cross-domain generalization.

Motivated by the urgent demands of AR/VR and digital twin applications for fast, generalizable, and deployment-friendly 3D reconstruction and novel view synthesis, this paper presents a systematic survey of feedforward deep learning methods—covering dominant representations including point clouds, 3D Gaussian splatting, and neural radiance fields—and focuses on three key challenges: pose-free input, dynamic scene modeling, and 3D-aware content generation. We propose the first unified taxonomy tailored to the feedforward paradigm, revealing inherent trade-offs between inference efficiency and cross-scene generalization. By integrating self-supervised learning, differentiable rendering, and multimodal input strategies—and leveraging standardized evaluation protocols and large-scale benchmarks—we comprehensively assess accuracy, latency, and robustness. Our analysis provides principled guidance and empirically grounded technology selection criteria for industrial-grade 3D vision systems.

This paper addresses the relative pose estimation problem for three calibrated cameras given only four correspondences across all views. To overcome limitations of conventional methods—namely, their reliance on more correspondences or insufficient robustness—we propose a novel strategy that approximates a fifth correspondence using the centroid of the four observed points. We further introduce the first joint three-view pose estimation framework integrating a 4-point affine fundamental matrix solver, a standard 5-point relative pose solver, and a P3P solver. Geometric modeling enhances robustness against noise and outliers, while local optimization refines accuracy. Evaluated on real-world datasets, our method achieves state-of-the-art performance: the centroid-based strategy significantly outperforms pure affine approaches, striking a superior balance among accuracy, robustness, and computational efficiency, with straightforward implementation.

This paper addresses the high computational complexity of two-view triangulation in multiview geometry. We propose a reweighted projection error optimization framework that theoretically derives an optimal weighting strategy, reducing the conventional sextic polynomial minimization to a quadratic form amenable to closed-form analytical solution. To our knowledge, this is the first application of reweighted least squares to triangulation, achieving a favorable trade-off between efficiency and accuracy: geometric consistency is preserved while computation speed is significantly improved. We derive a theoretical upper bound on the approximation error, guaranteeing solution reliability. Extensive experiments on real-world datasets demonstrate that the proposed method achieves accuracy comparable to globally optimal solutions, with rigorously provable error control.

This work addresses the limitations of existing panoramic stitching methods, which rely on pairwise feature matching and often fail to maintain multi-view geometric consistency in complex scenes characterized by weak textures, large disparities, or repetitive patterns, leading to misalignments and distortions. To overcome these challenges, the authors propose a photogrammetry-driven global alignment framework that leverages estimated camera poses to align images in 3D space. They introduce a novel 3D-aware Transformer architecture that explicitly models multi-view geometric consistency through joint feature optimization and cross-view information aggregation. Key contributions include the first formulation of multi-view consistency in 3D space, a Transformer-based 3D-aware stitching network, and the creation of the first large-scale real-world panoramic stitching dataset. Experiments demonstrate that the proposed method significantly outperforms state-of-the-art approaches in both alignment accuracy and visual quality, particularly exhibiting superior robustness and consistency in challenging scenarios.

This paper addresses model-free intrinsic camera calibration from a single unstructured outdoor image, without requiring gravity priors, camera-specific assumptions, or external geometric cues. We propose a pixel-ray regression-based intermediate representation that decouples intrinsic estimation into manifold-learning-driven ray modeling and model-agnostic closed-form inversion. The framework uniformly supports diverse projection models—including pinhole, Brown–Conrady, and Kannala–Brandt—and natively accommodates common image edits such as cropping and stretching. Our approach employs only a lightweight neural network, trained on orders-of-magnitude less data than 3D foundation models, yet consistently surpasses state-of-the-art methods in accuracy and robustness. Notably, it demonstrates superior cross-model generalization and resilience to edited images—key challenges in practical deployment.

In large-scale Structure-from-Motion (SfM), sparse inter-view overlap and drastic viewpoint changes—especially in aerial-to-ground scenarios—lead to low cross-image feature matching density and weak geometric consistency. To address this, we propose a geometry-guided hybrid matching paradigm: (1) geometric verification is formulated as an optimization problem based on Sampson distance; (2) detector-agnostic dense matching is fused with detector-driven sparse anchor guidance, where sparse anchors constrain and enhance the geometric consistency of dense matches; and (3) multi-view geometric consistency is explicitly modeled. Our method significantly improves both matching density and accuracy, outperforming state-of-the-art approaches in extreme large-scale settings. Consequently, camera pose estimation becomes more accurate, and the reconstructed 3D point cloud achieves higher completeness and fidelity.

This work addresses the frequent failure of multi-fisheye camera calibration due to poor initial intrinsic estimates, particularly under complex layouts and large fields of view where observation quality is degraded. The authors propose CO-Calib, a novel framework that reveals— for the first time—that limited radial span in observed features induces coupling between focal length and fisheye distortion parameters, leading to ill-conditioned updates. To mitigate this, they introduce a plug-and-play calibration data construction strategy integrating learning-based target detection, error-aware frame selection, multi-camera co-visibility constraints, and robust initialization. Experiments demonstrate that the method boosts calibration success rates from 68.1% to 99.3% on both synthetic and real multi-fisheye systems, substantially improving extrinsic accuracy and overall calibration stability.

Existing truth-free multi-view 3D consistency evaluation methods are prone to high-score hallucinations in the presence of artifacts, redundant views, or noise, making reliable geometric consistency assessment challenging. This work proposes a decomposable neural metric framework that decouples consistency evaluation into backbone, residual, and aggregation components, and integrates geometric verification signals from COLMAP—such as feature matching, registration, dense support, and reconstruction failure detection—to establish the first controllable robustness benchmark. Experiments demonstrate that the proposed metric significantly outperforms existing approaches on real novel-view synthesis results and human evaluations, achieving up to a fourfold increase in correlation with human judgment over MEt3R and a threefold improvement in robustness. This study also systematically reveals, for the first time, that prominent 3D foundation models—including VGGT, MASt3R, DUSt3R, and Fast3R—are susceptible to geometric hallucinations under irrelevant or degenerate inputs.

This work addresses the challenges of missing correspondences and incomplete geometric information in point cloud reconstruction from partially observed multi-view inputs. The authors propose a training-free optimization method that jointly recovers the 3D point cloud and its cross-view projection mappings. Built upon an extended multi-view synchronized embedding framework, the approach integrates variable projection, geometric constraints, and visibility modeling, making it applicable to both fixed and variable projection settings without requiring category-specific priors. Experiments on ShapeNet and Pix3D demonstrate that the method robustly reconstructs partial multi-view point clouds, consistently outperforming existing non-learning baselines across Chamfer distance, Earth Mover’s Distance (EMD), and Reconstruction Overlap Accuracy (ROA) metrics.

This work addresses the significant performance degradation of existing 3D reconstruction methods on non-pinhole imagery—such as fisheye or panoramic views—due to their reliance on the pinhole camera assumption. To overcome this limitation, we propose the first feed-forward, calibration-free, universal 3D reconstruction framework capable of handling diverse camera models. Our approach employs a dual-branch network to jointly estimate per-pixel ray directions and radial distances, complemented by a ray-aware global alignment mechanism that fuses local geometric cues while simultaneously optimizing pose and scale. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both 3D reconstruction and pose estimation across fisheye, panoramic, and pinhole image datasets, marking the first unified, calibration-free solution for cross-camera-model 3D reconstruction.

Existing camera calibration methods are often constrained by controlled environments or single-view assumptions, limiting their ability to model multi-view geometric consistency in real-world scenarios. This work proposes CalibAnyView, a unified framework that, for the first time, enables joint estimation of intrinsic camera parameters and gravity direction from an arbitrary number of input views. The approach integrates a multi-view Transformer, dense perspective field prediction, and geometric optimization, while introducing a heterogeneous lens distortion model. Alongside the method, the authors introduce the first large-scale multi-view outdoor video dataset. Experiments demonstrate that CalibAnyView outperforms state-of-the-art methods in both single- and multi-view settings, significantly enhancing accuracy and robustness in downstream tasks such as 3D reconstruction and robotic perception.