Sonar-MASt3R: Real-Time Opti-Acoustic Fusion in Turbid, Unstructured Environments

2026-03-13

AI Summary

This work addresses the challenge of robust real-time 3D reconstruction in turbid or low-light underwater environments, where optical cameras alone often fail. To overcome this limitation, the authors propose a hand-eye photometric-acoustic fusion system that, for the first time, enables MASt3R-based real-time dense reconstruction in real-world high-turbidity waters (0.5–12 NTU). The approach leverages MASt3R to extract dense correspondences from optical images and integrates geometric constraints provided by 3D sonar to significantly enhance both robustness and real-time performance. Evaluated in unstructured turbid settings, the method demonstrates superior accuracy and stability compared to existing baselines, establishing a new benchmark for underwater dense reconstruction under adverse visibility conditions.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: 3D Computer Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Underwater intervention is an important capability in several marine domains, with numerous industrial, scientific, and defense applications. However, existing perception systems used during intervention operations rely on data from optical cameras, which limits capabilities in poor visibility or lighting conditions. Prior work has examined opti-acoustic fusion methods, which use sonar data to resolve the depth ambiguity of the camera data while using camera data to resolve the elevation angle ambiguity of the sonar data. However, existing methods cannot achieve dense 3D reconstructions in real-time, and few studies have reported results from applying these methods in a turbid environment. In this work, we propose the opti-acoustic fusion method Sonar-MASt3R, which uses MASt3R to extract dense correspondences from optical camera data in real-time and pairs it with geometric cues from an acoustic 3D reconstruction to ensure robustness in turbid conditions. Experimental results using data recorded from an opti-acoustic eye-in-hand configuration across turbidity values ranging from <0.5 to >12 NTU highlight this method's improved robustness to turbidity relative to baseline methods.

Problem

• opti-acoustic fusion
• turbid underwater environment
• real-time 3D reconstruction
• sonar perception
• underwater intervention

Innovation

• opti-acoustic fusion
• real-time 3D reconstruction
• sonar
• turbid underwater environment
• MASt3R