Evaluating the Efficacy of Cut-and-Paste Data Augmentation in Semantic Segmentation for Satellite Imagery

2024-04-08

IEEE International Geoscience and Remote Sensing Symposium

AI Summary

To address label scarcity, class imbalance, and scene complexity in satellite image semantic segmentation, this paper proposes a pixel-level Cut-and-Paste data augmentation method. Unlike conventional instance-aware approaches requiring explicit instance masks, our method automatically extracts semantic instances via connected-component analysis and randomly pastes them into target images—eliminating the need for additional instance-level annotations. This represents the first effective adaptation of instance-based Cut-and-Paste augmentation to semantic segmentation. Evaluated on the DynamicEarthNet dataset with a U-Net backbone, our approach achieves a mean Intersection-over-Union (mIoU) of 44.1%, outperforming the baseline by 6.2 percentage points. The method significantly improves model generalization and enhances discrimination capability for small objects and long-tail classes.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Satellite imagery is crucial for tasks like environmental monitoring and urban planning. Typically, it relies on semantic segmentation or Land Use Land Cover (LULC) classification to categorize each pixel. Despite the advancements brought about by Deep Neural Networks (DNNs), their performance in segmentation tasks is hindered by challenges such as limited availability of labeled data, class imbalance and the inherent variability and complexity of satellite images. In order to mitigate those issues, our study explores the effectiveness of a Cut-and-Paste augmentation technique for semantic segmentation in satellite images. We adapt this augmentation, which usually requires labeled instances, to the case of semantic segmentation. By leveraging the connected components in the semantic segmentation labels, we extract instances that are then randomly pasted during training. Using the DynamicEarthNet dataset and a U-Net model for evaluation, we found that this augmentation significantly enhances the mIoU score on the test set from 37.9 to 44.1. This finding highlights the potential of the Cut-and-Paste augmentation to improve the generalization capabilities of semantic segmentation models in satellite imagery.

Problem

• Satellite Image Augmentation
• Semantic Segmentation
• Land Use Classification

Innovation

• Data Augmentation
• Semantic Segmentation
• Satellite Imagery