ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Object

2025-03-15

AI Summary

To address low interactive segmentation accuracy for small moving targets in remote sensing videos—caused by their minute scale, ambiguous features, and poor model generalization—this paper proposes an efficient optimization framework built upon the Segment Anything Model (SAM). The method introduces three key innovations: (1) a LoRA-based lightweight fine-tuning strategy to improve adaptation efficiency; (2) a deep feature enhancement mechanism to strengthen discriminative representations of small targets; and (3) a novel decoder integrating global contextual information with local boundary cues. Coupled with multi-scale training and context-aware mask decoding, the approach achieves a 13% improvement in IoU on remote sensing video benchmarks; an additional 6% gain is obtained via optimized data pipelining. Notably, in zero-shot transfer to tracking tasks, the generated masks attain quality comparable to human annotations.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The availability of large-scale remote sensing video data underscores the importance of high-quality interactive segmentation. However, challenges such as small object sizes, ambiguous features, and limited generalization make it difficult for current methods to achieve this goal. In this work, we propose ROS-SAM, a method designed to achieve high-quality interactive segmentation while preserving generalization across diverse remote sensing data. The ROS-SAM is built upon three key innovations: 1) LoRA-based fine-tuning, which enables efficient domain adaptation while maintaining SAM's generalization ability, 2) Enhancement of deep network layers to improve the discriminability of extracted features, thereby reducing misclassifications, and 3) Integration of global context with local boundary details in the mask decoder to generate high-quality segmentation masks. Additionally, we design the data pipeline to ensure the model learns to better handle objects at varying scales during training while focusing on high-quality predictions during inference. Experiments on remote sensing video datasets show that the redesigned data pipeline boosts the IoU by 6%, while ROS-SAM increases the IoU by 13%. Finally, when evaluated on existing remote sensing object tracking datasets, ROS-SAM demonstrates impressive zero-shot capabilities, generating masks that closely resemble manual annotations. These results confirm ROS-SAM as a powerful tool for fine-grained segmentation in remote sensing applications. Code is available at https://github.com/ShanZard/ROS-SAM.

Problem

• Achieves high-quality interactive segmentation for remote sensing moving objects.
• Addresses challenges like small object sizes and ambiguous features in segmentation.
• Enhances generalization across diverse remote sensing data using innovative techniques.

Innovation

• LoRA-based fine-tuning for domain adaptation
• Enhanced deep network layers for feature discriminability
• Global-local context integration in mask decoder