surface normal estimation

Neural implicit methods struggle to reconstruct fine-grained geometry, sharp edges, and thin structures from sparse multi-view RGB inputs—particularly with only two views (front and back). Method: Moving beyond conventional zero-order geometric constraints (e.g., point-projection consistency), we introduce first-order differential constraints—specifically surface normals—as explicit supervision for neural implicit modeling. We estimate monocular depth using Depth Anything and derive approximate image-space surface normals, formulating a normal consistency loss that jointly optimizes the first-order differentiable properties of signed distance functions (SDFs) or NeRF-like implicit fields. Results: Evaluated on both synthetic and real-world datasets, our method achieves high-fidelity 3D surface reconstruction from merely two RGB images. It significantly outperforms state-of-the-art approaches in PSNR, Chamfer distance, and visual quality, demonstrating that normal supervision is critical for recovering fine-scale geometric details.

Estimating surface normals from noisy point clouds remains a longstanding challenge, as existing methods rely heavily on clean inputs and strong supervision, failing to simultaneously preserve geometric details and ensure robustness. This paper proposes a local gradient-aware implicit surface filtering framework: it guides point cloud projection onto the underlying surface via an implicit function and introduces a joint normal-distance operator to enforce geometric constraints. To mitigate gradient degradation, we design a local gradient consistency loss; to suppress over-smoothing, we develop a multi-scale directional aggregation mechanism. Crucially, our method performs end-to-end learning without explicit normal supervision, unifying denoising, normal estimation, and surface reconstruction. Extensive experiments demonstrate state-of-the-art performance across multiple benchmarks, significantly improving fine-detail recovery under complex noise patterns. The code and pre-trained models are publicly available.

To address inaccurate surface normal estimation in autonomous driving caused by sparse, non-uniform, and noisy LiDAR point clouds, this paper proposes the Hybrid Geometric Transformer (HGT). HGT is the first Transformer-based architecture to jointly model visual semantics and 3D geometric priors. It introduces a cross-modal feature alignment mechanism and a multi-modal self-supervised strategy to enable efficient geometric knowledge transfer from simulation to real-world scenes (e.g., KITTI). By fusing LiDAR point clouds with camera images through joint encoding and domain-adaptive fine-tuning, HGT significantly enhances geometric perception robustness. On KITTI, it achieves state-of-the-art accuracy in surface normal estimation—outperforming prior methods—and effectively supports downstream tasks such as collision avoidance and occlusion reasoning. Experimental results demonstrate strong generalization and practical applicability.

Reconstructing multiple objects from a single RGB-D frame faces core challenges including geometric unobservability in occluded regions, poor noise robustness, weak generalization, and miscalibrated confidence estimation. Method: We propose a retrieval-augmented Bayesian reconstruction framework. It introduces, for the first time, a cross-modal part-level retrieval mechanism that dynamically matches semantically similar parts from a pre-trained 3D mesh library to construct an information-rich, adaptive Bayesian prior—jointly modeling geometric uncertainty and occlusion reasoning. The framework integrates RGB-D perception, differentiable rendering, and probabilistic inference to yield well-calibrated reconstruction confidence. Contribution/Results: Evaluated on both synthetic and real-world cluttered scenes, our method significantly improves reconstruction accuracy in occluded regions and enhances robustness to sensor noise. Crucially, it enables dexterous robotic hands to achieve high-success-rate grasping, demonstrating practical applicability in complex, unstructured environments.

This work addresses the problem of open-domain single-image 3D geometric reconstruction. To resolve global scale and translation ambiguities, we propose an affine-invariant 3D point cloud representation. Methodologically, we design an optimal point cloud alignment solver and a multi-scale local geometric consistency loss to mitigate the inherent ambiguity of monocular geometric supervision. Our approach integrates affine-invariant representation learning, robust point cloud registration, and end-to-end training on a hybrid large-scale dataset. Extensive experiments demonstrate that our method achieves state-of-the-art performance across multiple unseen benchmarks. It significantly improves accuracy and generalization in monocular 3D point cloud reconstruction, depth estimation, and field-of-view prediction. By eliminating the need for camera calibration or explicit metric priors, our framework establishes a new paradigm for uncalibrated single-image geometric understanding.

This work investigates whether VGGT (Vision Geometry Foundation Transformer), trained without explicit geometric supervision, implicitly acquires an understanding of camera geometry and scene structure—not merely relying on appearance-driven data priors. To this end, we conduct a systematic analysis of its internal representations via feature-space probing, attention-pattern visualization, input-space masking and perturbation experiments, and comparative evaluation against traditional multi-stage geometric pipelines. We establish, for the first time, that a purely data-driven, single-stage Transformer can spontaneously emerge geometric reasoning capabilities; specifically, its global self-attention layers implicitly perform cross-view correspondence matching and encode epipolar geometric constraints. Experiments demonstrate VGGT’s strong geometric robustness under occlusion, appearance variation, and camera configuration changes. Moreover, its implicit geometric understanding synergizes with data priors to enhance 3D perception performance.

This work addresses the challenge in multi-view normal fusion where varying camera distances cause inconsistent spatial scales of pixel coverage, leading to conflicting normals and loss of high-frequency geometric details. To resolve this, the authors propose a scale-encoded neural surface representation that explicitly models pixel coverage scale within a neural implicit framework for the first time. Each 3D point is associated with a local spatial scale, and scale-aware normals are computed via a hybrid grid encoding scheme. Furthermore, a scale-aware mesh extraction module is introduced to assign each vertex an optimal local scale based on observed data. The method significantly outperforms existing approaches under multi-distance capture conditions, effectively preserving normal consistency while achieving high-fidelity surface reconstruction with enhanced retention of fine-scale geometric details.

This work addresses the limitations of existing feedforward monocular 3D reconstruction methods in capturing fine-grained local surface geometry, which conventional evaluation metrics often fail to adequately quantify. To this end, we propose SurGe, a novel framework that introduces a point-wise surface normal-based metric to explicitly assess local geometric orientation errors. SurGe incorporates several key innovations, including a point gradient matching loss, a neighborhood-aware attention decoder, depth-normalized 3D finite differences, and a feature-progressive upsampling strategy. Extensive experiments demonstrate that SurGe achieves state-of-the-art performance on eight zero-shot monocular geometry benchmarks in terms of the global point-wise AbsRel metric, while consistently and significantly improving the fidelity of both local surface geometry and its corresponding normals.

This work addresses the challenge of unreliable disparity and surface normal estimation in complex scenarios—such as low-light conditions, highly reflective surfaces, and transparent objects—where existing feedforward stereo matching methods suffer from insufficient geometric priors. To overcome this limitation, we propose GeoStereo, a novel framework that introduces diffusion models to stereo geometric estimation for the first time. GeoStereo establishes a bidirectional guidance mechanism between disparity and surface normals through disparity-guided normal initialization and warping-based left-view alignment conditioning, enabling their joint optimization. Evaluated under unsupervised settings, our method significantly enhances geometric reconstruction robustness, achieving state-of-the-art zero-shot disparity estimation performance on KITTI and NYUv2, and delivering leading surface normal accuracy on real-world indoor benchmarks including iBims-1 and ScanNet.

Existing feedforward 3D reconstruction methods are largely confined to 2.5D representations of visible surfaces, struggling to efficiently recover complete geometric structures—particularly in occluded regions. This work proposes a unified feedforward framework based on sparse 3D queries, formulating 3D reconstruction for the first time as a sparse query inference problem. By introducing explicit geometric anchors as queries in global 3D space, the method leverages a decoupled cross-attention mechanism to enable multi-view feature interaction and employs differentiable rendering with 3D Gaussians. The approach reconstructs complete scenes—including occluded areas—in a single forward pass, significantly outperforming existing feedforward methods on the Mip-NeRF 360 and VR-NeRF datasets with orders-of-magnitude fewer primitives while achieving state-of-the-art results in both rendering quality and geometric accuracy.