RGBD-to-3D Object Mesh Refinement via Depth Matching and Symmetry Propagation

2026-10-07

AI Summary

This work addresses the view-inconsistency problem in single-view 3D reconstruction caused by depth discontinuities and self-occluded regions, proposing a lightweight, plug-and-play correction method. The approach employs a fully closed-form solving pipeline that achieves depth alignment via bipartite graph matching to rectify visible surfaces, and leverages symmetry plane detection to mirror geometric information toward occluded regions. Consequently, its computational efficiency surpasses optimization-based test-time refinement methods by several orders of magnitude. Evaluated on the GSO and OmniObject3D datasets, the proposed method yields consistent performance gains across five backbone networks, outperforming existing refinement approaches while demonstrating effective transferability to noisy real-world data.

Abstract

Single-view 3D reconstructors often produce plausible meshes that disagree with the input view, especially near depth discontinuities and self-occlusions. We present a lightweight, plug-and-play RGBD-to-3D refinement that improves any RGB-to-3D reconstructor without retraining. Given a depth map, we correct the visible surface by bipartite matching to back-projected depth points, mirror these corrections onto the occluded side across a detected symmetry plane, and propagate them with a smoothness solver. Every stage is closed-form, making the method orders of magnitude faster than optimization-heavy test-time refinement. On GSO and OmniObject3D with five backbones, it yields consistent gains, also with monocular pseudo-depth, benefits more from symmetry on symmetric objects, and compares favorably with prior refinement in accuracy and runtime. It further improves an RGB-D-to-mesh reconstructor and transfers to real captures with noisy sensor depth.

Problem

• 3D reconstruction
• mesh refinement
• depth discontinuities
• self-occlusions
• RGBD

Innovation

• RGBD-to-3D refinement
• depth matching
• symmetry propagation
• plug-and-play
• closed-form solver