HierGF: Hierarchical Gaussian Fields via Geometry-perception Message Passing for Sparse-view 3D Reconstruction

2026-10-01

AI Summary

This study addresses the challenges of establishing multi-view consistency and recovering missing structures in under-sampled regions under sparse-view settings. To this end, we propose a 3D reconstruction framework based on hierarchical Gaussian fields. Methodologically, we design a two-stage geometry-aware backbone network with a message-passing mechanism to fuse coarse 3D geometry and 2D generative priors into structured pseudo-supervision. Furthermore, a learnable confidence network is introduced to guide gradient optimization, alongside a geometry-consistent densification module that eliminates reliance on heuristic density control. This work significantly improves multi-view alignment accuracy under sparse inputs, effectively mitigates blurring and structural holes in under-sampled regions, and enhances overall reconstruction robustness.

Abstract

Sparse view 3D reconstruction is an important and common scenario in multimedia applications, such as augmented reality/virtual reality (AR/VR) content creation, cultural heritage digitization, and certain robotic applications, where only a limited number of randomly captured views may be available. However, sparse views contain only limited 3D information, posing two major challenges:1) too few images are available for matching, making it difficult to build multi-view consistency; 2) insufficient view coverage leads to a lack of information in under-sampled regions, resulting in missing parts of object structure. Existing methods mostly still rely on limited reprojection errors and regularization terms, which are prone to overfitting to a single view and inconsistent appearances across views. In geometrically under-sampled regions, they often rely on heuristic density control, lacking reliable guidance and often resulting in blurring and structural holes.To address these issues, this paper proposes Hierarchical Gaussian Fields (HierGF), which revisits sparse-view reconstruction from a hierarchical geometry-perception perspective and converts limited observations into reliable self-generated supervision beyond fixed priors and heuristic density control. In particular, we transform coarse 3D geometric information and additional 2D generative priors into structured pseudo-supervision through a two-stage geometry-perception backbone network, thereby enhancing multi-view consistency with very few input views. In addition, we introduce a learnable confidence network to guide gradients toward cross-view consistent content, and a geometrically consistent densification module to improve the reconstruction of multi-view alignment and under-sampled regions.

Problem

• Sparse-view 3D Reconstruction
• Multi-view Consistency
• Under-sampled Regions
• Geometry-perception

Innovation

• Sparse-view 3D Reconstruction
• Hierarchical Gaussian Fields
• Geometry-perception Message Passing
• Pseudo-supervision
• Learnable Confidence Network