Frame-to-Panorama Localization and Context-Aware Sampling for Scene-Specific Ship Detection in a Smart Marina Testbed

2026-09-24

AI Summary

This study addresses the challenge of efficiently constructing scenario-specific ship detection datasets from historical PTZ videos in smart ports, where metadata is often missing. To this end, it proposes an end-to-end data curation framework that integrates environmental context with visual diversity. By combining panoramic localization, SuperPoint/LightGlue feature matching, and meteorological metadata extraction with Gaussian Mixture Model clustering and a quadratic sampling strategy, the approach transforms redundant video footage into a compact yet diverse training subset. The proposed method achieves a 99.5% frame compression rate while attaining 94.78% AP50 by annotating only 220 frames. This significantly reduces annotation costs without compromising detection accuracy, thereby establishing a new paradigm for data selection in long-range object detection tasks.

Abstract

Smart maritime infrastructures provide continuous access to heterogeneous sensing streams, enabling repeated experimentation, digital-twin development, and AI-based maritime services. However, sensing hardware alone is not sufficient for scene-specific model development: historical video streams must also be spatially indexed, contextualized, and reduced to informative subsets for annotation. This paper presents a frame-to-panorama localization and context-aware sampling pipeline for ship detection in historical PTZ maritime video lacking reliable pan, tilt, and zoom metadata. The main contribution is an end-to-end data-curation approach that recovers camera-view information from historical PTZ video and combines it with environmental context and visual diversity to construct compact, scene-specific training sets. Specifically, frames are localized on a reference panorama using SuperPoint and LightGlue, enriched with weather and solar-state metadata, and selected through diversity sampling to preserve variation across camera view and environmental conditions. A second context-aware stage targets under-represented distant-vessel cases near the horizon using tile-level visual embeddings and Gaussian Mixture Model clustering. Applied within the CMMI MDigi-I Smart Marina testbed, the proposed pipeline reduces 40,718 candidate frames to 220 images for annotation, corresponding to a 99.5% reduction. A YOLO26-m detector fine-tuned on this subset achieves a mean AP50 of 94.78% $\pm$ 0.51% and a mean AP50-95 of 75.10% $\pm$ 1.73% under sequence-grouped five-fold cross-validation. These results demonstrate that highly redundant infrastructure video streams can be transformed into compact, spatially and contextually diverse training sets for scene-specific detector adaptation while substantially reducing annotation effort.

Problem

• ship detection
• data curation
• smart marina
• PTZ video
• scene-specific adaptation

Innovation

• Frame-to-Panorama Localization
• Context-Aware Sampling
• Data Curation
• Ship Detection
• Smart Marina