Convolutional Neural Shading for High-Quality 3D Reconstruction from Multi-View Images

2026-07-30

AI Summary

Existing neural rendering approaches rely on single-point geometric information, making it difficult to recover fine geometric details in dark or textureless regions. To address this limitation, this work proposes a Convolutional Neural Shading (CNS) framework that introduces a convolutional neural shader to model local geometric cues from multi-view images and integrates a refined displacement network that leverages spatial neighborhood information to optimize surface geometry. By incorporating differentiable rendering and explicitly modeling multi-view geometric consistency, the method significantly outperforms current state-of-the-art techniques in both reconstruction accuracy and rendering quality, effectively recovering intricate geometric details while reducing surface irregularities.

Abstract

We propose a convolutional neural shading (CNS), a novel pipeline to reconstruct high-quality 3D shapes from multi-view images. Several recent studies have used neural radiance fields and other neural differentiable rendering methods to understand 3D geometry. However, these approaches rely on single-point geometric information, such as positions and normals of the surface, leading to a lack of detailed local geometry. Our approach addresses the inherent limitations of single-point information by leveraging a neural shader to capture variations even in dark and textureless regions with a convolutional neural shader, resulting in far more accurate geometry predictions. Additionally, our method mitigates surface irregularities at image boundaries by introducing a fine-detail displacement network, which utilizes spatial information of surface geometry and learns fine displacement details by correlating neighboring values in the rendering coordinates. Through extensive experiments, our proposed method has demonstrated significant quality improvements in the reconstructed shapes and rendered images over current state-of-the-art methods.

Problem

• 3D reconstruction
• multi-view images
• local geometry detail
• surface irregularities
• neural rendering

Innovation

• Convolutional Neural Shading
• Multi-View 3D Reconstruction
• Neural Rendering
• Fine-Detail Displacement
• Differentiable Rendering