Synthetic-to-real domain bridging for single-view 3D reconstruction of ships for maritime monitoring

2025-09-16

Optical Engineering + Applications

AI Summary

This work proposes an end-to-end single-view 3D ship reconstruction framework trained exclusively on synthetic data, addressing the limitations of existing methods that rely on multi-view imagery, real 3D annotations, or high computational costs—constraints that hinder real-time maritime surveillance. The approach achieves cross-domain generalization without any real 3D supervision by integrating a Splatter Image network based on sparse 3D Gaussian representations, a YOLOv8 segmentation module, and homography-based georeferencing, further enhanced with AIS metadata for geographic alignment and interactive visualization. Experiments demonstrate high-fidelity reconstructions on synthetic data and strong cross-domain performance on real-world maritime imagery (ShipSG), enabling efficient and interactive 3D ship monitoring in practical scenarios.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Three-dimensional (3D) reconstruction of ships is an important part of maritime monitoring, allowing improved visualization, inspection, and decision-making in real-world monitoring environments. However, most state-of-the-art 3D reconstruction methods require multi-view supervision, annotated 3D ground truth, or are computationally intensive, making them impractical for real-time maritime deployment. In this work, we present an efficient pipeline for single-view 3D reconstruction of real ships by training entirely on synthetic data and requiring only a single view at inference. Our approach uses the Splatter Image network, which represents objects as sparse sets of 3D Gaussians for rapid and accurate reconstruction from single images. The model is first fine-tuned on synthetic ShapeNet vessels and further refined with a diverse custom dataset of 3D ships, bridging the domain gap between synthetic and real-world imagery. We integrate a state-of-the-art segmentation module based on YOLOv8 and custom preprocessing to ensure compatibility with the reconstruction network. Postprocessing steps include real-world scaling, centering, and orientation alignment, followed by georeferenced placement on an interactive web map using AIS metadata and homography-based mapping. Quantitative evaluation on synthetic validation data demonstrates strong reconstruction fidelity, while qualitative results on real maritime images from the ShipSG dataset confirm the potential for transfer to operational maritime settings. The final system provides interactive 3D inspection of real ships without requiring real-world 3D annotations. This pipeline provides an efficient, scalable solution for maritime monitoring and highlights a path toward real-time 3D ship visualization in practical applications.

Problem

• single-view 3D reconstruction
• synthetic-to-real domain gap
• maritime monitoring
• ship 3D modeling
• real-time 3D visualization

Innovation

• single-view 3D reconstruction
• synthetic-to-real domain adaptation
• 3D Gaussian splatting
• maritime monitoring
• domain bridging