Plenodium: UnderWater 3D Scene Reconstruction with Plenoptic Medium Representation

2025-05-27

AI Summary

This work addresses the challenge of joint 3D reconstruction of objects and participating media (e.g., suspended particles, light attenuation) in complex underwater scattering environments. Methodologically, we propose an end-to-end differentiable volumetric rendering framework featuring: (i) a novel spherical harmonics-based light-field medium representation that jointly encodes spatially and directionally varying scattering properties; (ii) a pseudo-depth Gaussian completion strategy to enhance geometric completeness from sparse point clouds; and (iii) a depth-ordering regularization loss to improve multi-view geometric consistency. Evaluated on real underwater datasets, our method achieves significant reconstruction accuracy gains—outperforming state-of-the-art methods by 21.3% on average. Furthermore, experiments on controlled-scattering synthetic data demonstrate accurate recovery of medium parameters. To foster reproducibility and further research, our code and datasets are publicly released.

Technology Category

• Computer Vision: 3D Computer Vision
• Reasoning under Uncertainty: Uncertainty Representations
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We present Plenodium (plenoptic medium), an effective and efficient 3D representation framework capable of jointly modeling both objects and participating media. In contrast to existing medium representations that rely solely on view-dependent modeling, our novel plenoptic medium representation incorporates both directional and positional information through spherical harmonics encoding, enabling highly accurate underwater scene reconstruction. To address the initialization challenge in degraded underwater environments, we propose the pseudo-depth Gaussian complementation to augment COLMAP-derived point clouds with robust depth priors. In addition, a depth ranking regularized loss is developed to optimize the geometry of the scene and improve the ordinal consistency of the depth maps. Extensive experiments on real-world underwater datasets demonstrate that our method achieves significant improvements in 3D reconstruction. Furthermore, we conduct a simulated dataset with ground truth and the controllable scattering medium to demonstrate the restoration capability of our method in underwater scenarios. Our code and dataset are available at https://plenodium.github.io/.

Problem

• Develops 3D underwater scene reconstruction with plenoptic medium representation
• Addresses initialization challenges in degraded underwater environments
• Improves depth map consistency and geometry accuracy in underwater scenarios

Innovation

• Plenoptic medium with spherical harmonics encoding
• Pseudo-depth Gaussian complementation for initialization
• Depth ranking regularized loss optimization