🤖 AI Summary
Traditional SfM/MVS methods suffer from complex pipelines, high computational cost, and poor robustness in textureless regions. To address these limitations, this paper systematically reviews feedforward, single-pass 3D reconstruction techniques—particularly end-to-end models such as DUSt3R—and proposes a Transformer-based paradigm for joint pose and geometry estimation, eliminating iterative optimization and unifying multi-view geometry and camera pose estimation. The method supports arbitrary numbers of input images, ensuring strong generalization and flexibility. By integrating correspondence modeling, multi-view feature fusion, and joint regression, it achieves superior efficiency and robustness over both classical approaches and learning-based methods (e.g., MVSNet) on standard benchmarks. We further survey prevailing architectural frameworks, benchmark datasets, and evaluation metrics, and identify key future directions—including dynamic scene modeling and model scalability.
📝 Abstract
3D reconstruction, which aims to recover the dense three-dimensional structure of a scene, is a cornerstone technology for numerous applications, including augmented/virtual reality, autonomous driving, and robotics. While traditional pipelines like Structure from Motion (SfM) and Multi-View Stereo (MVS) achieve high precision through iterative optimization, they are limited by complex workflows, high computational cost, and poor robustness in challenging scenarios like texture-less regions. Recently, deep learning has catalyzed a paradigm shift in 3D reconstruction. A new family of models, exemplified by DUSt3R, has pioneered a feed-forward approach. These models employ a unified deep network to jointly infer camera poses and dense geometry directly from an Unconstrained set of images in a single forward pass. This survey provides a systematic review of this emerging domain. We begin by dissecting the technical framework of these feed-forward models, including their Transformer-based correspondence modeling, joint pose and geometry regression mechanisms, and strategies for scaling from two-view to multi-view scenarios. To highlight the disruptive nature of this new paradigm, we contrast it with both traditional pipelines and earlier learning-based methods like MVSNet. Furthermore, we provide an overview of relevant datasets and evaluation metrics. Finally, we discuss the technology's broad application prospects and identify key future challenges and opportunities, such as model accuracy and scalability, and handling dynamic scenes.