VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation

2026-09-29

AI Summary

This study addresses the limitations of single-modality optical remote sensing for ship perception under complex sea conditions and the absence of large-scale multimodal benchmarks. To this end, we construct the world's largest multimodal ship perception benchmark dataset, which integrates heterogeneous optical and synthetic aperture radar (SAR) sensor data. Comprising 800,000 finely annotated image pairs spanning multiple resolutions and complex scenarios, the dataset supports multi-task evaluation including object detection, counting, and density estimation. As the first global-scale multimodal ship benchmark, this work systematically evaluates state-of-the-art models and reveals cross-modal complementary cues, effectively overcoming single-modality perception bottlenecks. The dataset has been fully open-sourced to significantly advance academic research and community development in maritime traffic understanding.

Abstract

Vessel perception from space is crucial for a wide range of maritime applications, from traffic monitoring to environmental protection. However, most existing datasets predominantly focus on general object detection tasks in optical remote sensing (RS) images. Relying solely on single-modality optical RS images proves inadequate for effectively perceiving vessel objects in complex maritime scenarios, where ever-changing weather conditions (e.g., clouds and rain), the need for day-and-night coverage, and the inherent limitations of a single imaging modality pose significant challenges. To fill this gap, we introduce VesselBench-800K, the largest-to-date benchmark dataset on a global scale for vessel perception in multimodal RS images. As its name suggests, VesselBench-800K comprises 800,000 images, each at a resolution of 512x512 pixels, specifically curated for vessel perception tasks such as detection, counting, and density estimation. These multimodal image pairs (i.e., optical, SAR) are collected from diverse platforms, sensors, scenes, shooting heights, and synthetic sources, spanning spatial resolutions from 4.5m to 0.1m. Furthermore, we evaluate numerous state-of-the-art detection, counting, and density estimation models on VesselBench-800K through both qualitative and quantitative comparisons. By revealing previously unrecognized cues, this dataset holds immense potential to significantly advance our understanding of marine traffic. Our VesselBench dataset will be publicly available at https://github.com/danfenghong/IEEE_TGRS_VesselBench to support and contribute to community development.

Problem

• Vessel perception
• Multimodal remote sensing
• Object detection
• Density estimation
• Maritime scenarios

Innovation

• Multimodal Remote Sensing
• Vessel Detection
• Density Estimation
• Benchmark Dataset
• SAR and Optical