🤖 AI Summary
This work addresses geometric inconsistencies, error accumulation, and poor scalability in real-time multi-view 3D reconstruction caused by the tight coupling of extrinsic calibration, point cloud fusion, and global optimization. To overcome these limitations, we propose FUSE-Flow, a novel framework that decouples calibration and fusion into two synergistic modules for the first time. The GMAC module leverages geometric constraints and a multi-view reconstruction Transformer to estimate sparse extrinsics without requiring calibration targets, while the FUSE module enables stateless, real-time point cloud fusion through confidence-weighted integration and adaptive spatial hashing. These modules mutually refine each other via a confidence feedback mechanism. Extensive experiments on public datasets and real-world systems demonstrate that our approach significantly outperforms existing methods in accuracy, dynamic stability, and scalability, enabling large-scale real-time 3D reconstruction.
📝 Abstract
Real-time multi-camera 3D reconstruction is a key foundation for immersive media, remote interaction and spatial computing. While synchronized camera arrays are widely adopted, achieving geometrically consistent and scalable real-time reconstruction remains challenging. A key challenge is the close linkage among extrinsic calibration, multi-view fusion and global optimization, which causes fluctuating reconstruction results, cumulative errors and poor system expandability. We propose a decoupled framework for calibration and stateless real-time multi-view point cloud fusion (FUSE-Flow), a framework with two collaborative components: geometry-aligned multi-view extrinsic calibration (GMAC) and reliability-guided multi-view point cloud fusion (FUSE). This split design avoids conflicting optimization objectives for targeted improvement. The GMAC module refines camera extrinsics via geometric constraints and multi-view reconstruction transformers, enabling accurate sparse-view calibration without calibration targets, dense images or global bundle adjustment. The FUSE module integrates confidence weighting and adaptive spatial hashing for stateless fusion, ensuring linear time and memory consumption. The two modules mutually reinforce each other: accurate camera poses boost fusion accuracy, and confidence-aware fusion corrects calibration biases. Validated on public datasets and real camera setups, FUSE-Flow outperforms mainstream real-time reconstruction methods in visual effect, dynamic stability and scalability, providing a practical solution for large-scale real-time 3D reconstruction.