Revisiting Multi-View Stereo: A Sequence-to-Sequence Formulation

2026-09-21

AI Summary

本文针对多视图立体视觉中几何失真的问题，提出了一种序列到序列的方法，通过全局变换架构和相机参数嵌入来提高3D重建的准确性。

Abstract

Computing accurate geometry from multi-view images is a fundamental problem in computer vision. Recent feed-forward (FF) models jointly estimate 3D geometry and camera parameters, but they typically suffer from geometry distortion caused by reconstruction ambiguity, even when ground-truth camera parameters are supplied. In this paper, we study the multi-view stereo (MVS) problem with known camera parameters and propose a novel approach that bridges conventional MVS and FF methods. Rather than casting MVS as a sequence-to-one mapping that predicts depth only for a single reference view, we reformulate it as a sequence-to-sequence task, akin to FF models, that jointly predicts geometry for all input views. We introduce a global transformer-based architecture with two components that explicitly exploit camera-induced priors: ray-map embeddings that inject camera parameters into image patch tokens, making the transformer camera-aware, and a unified global cost volume that replaces conventional per-view cost volumes to jointly capture 3D structure across all views. Extensive experiments on multiple public benchmarks show our approach achieves state-of-the-art performance, surpassing both MVS and FF reconstruction baselines.

Problem

• multi-view stereo
• geometry distortion
• reconstruction ambiguity

Innovation

• sequence-to-sequence
• global transformer
• ray-map embeddings
• unified global cost volume