Graph Learning-Driven Multi-Vessel Association: Fusing Multimodal Data for Maritime Intelligence

2025-04-12

AI Summary

This paper addresses the multi-ship association challenge in high-density waterways under multimodal AIS and CCTV data fusion, characterized by heterogeneous modal dimensions, dynamic target cardinality mismatches, scale variations, occlusions, and asynchronous sampling. We propose an end-to-end graph neural network framework. Key contributions include: (1) a novel co-designed temporal graph attention and spatiotemporal attention mechanism to model cross-modal dynamic spatiotemporal dependencies; (2) an uncertainty-aware MLP fusion module that enhances robust similarity estimation under sparse, imbalanced, and high-density conditions; and (3) differentiable Hungarian optimization for globally optimal association. Evaluated on a real-world maritime dataset, our method significantly outperforms state-of-the-art approaches—particularly under degraded AIS/CCTV data quality or heavy traffic—demonstrating superior association accuracy and robustness.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Ensuring maritime safety and optimizing traffic management in increasingly crowded and complex waterways require effective waterway monitoring. However, current methods struggle with challenges arising from multimodal data, such as dimensional disparities, mismatched target counts, vessel scale variations, occlusions, and asynchronous data streams from systems like the automatic identification system (AIS) and closed-circuit television (CCTV). Traditional multi-target association methods often struggle with these complexities, particularly in densely trafficked waterways. To overcome these issues, we propose a graph learning-driven multi-vessel association (GMvA) method tailored for maritime multimodal data fusion. By integrating AIS and CCTV data, GMvA leverages time series learning and graph neural networks to capture the spatiotemporal features of vessel trajectories effectively. To enhance feature representation, the proposed method incorporates temporal graph attention and spatiotemporal attention, effectively capturing both local and global vessel interactions. Furthermore, a multi-layer perceptron-based uncertainty fusion module computes robust similarity scores, and the Hungarian algorithm is adopted to ensure globally consistent and accurate target matching. Extensive experiments on real-world maritime datasets confirm that GMvA delivers superior accuracy and robustness in multi-target association, outperforming existing methods even in challenging scenarios with high vessel density and incomplete or unevenly distributed AIS and CCTV data.

Problem

• Fusing multimodal data for maritime vessel association
• Overcoming dimensional disparities in AIS and CCTV data
• Improving accuracy in dense vessel traffic scenarios

Innovation

• Graph learning-driven multi-vessel association method
• Integrates AIS and CCTV with graph neural networks
• Uses temporal and spatiotemporal attention mechanisms