Self-Supervised Learning for Image Segmentation: A Comprehensive Survey

2025-05-19

AI Summary

Self-supervised learning (SSL) for image segmentation often relies heavily on large-scale annotated data, hindering its practical deployment. This paper presents a comprehensive survey of over 150 state-of-the-art SSL-based segmentation works published between 2018 and 2024. Methodologically, it introduces the first three-dimensional taxonomy—spanning pretraining paradigms (e.g., contrastive learning, generative modeling, redundancy reduction, geometric transformation), downstream tasks (semantic, instance, and medical segmentation), and benchmark datasets—and establishes a reproducible, unified evaluation framework to uncover methodological commonalities and performance limits. Furthermore, it constructs a knowledge graph integrating methodologies, datasets, benchmark results, and critical limitations analysis. The contributions significantly lower the entry barrier for researchers, provide theoretical foundations for standardization, and offer practical guidelines for real-world adoption of SSL in segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Unsupervised & Self-Supervised Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Supervised learning demands large amounts of precisely annotated data to achieve promising results. Such data curation is labor-intensive and imposes significant overhead regarding time and costs. Self-supervised learning (SSL) partially overcomes these limitations by exploiting vast amounts of unlabeled data and creating surrogate (pretext or proxy) tasks to learn useful representations without manual labeling. As a result, SSL has become a powerful machine learning (ML) paradigm for solving several practical downstream computer vision problems, such as classification, detection, and segmentation. Image segmentation is the cornerstone of many high-level visual perception applications, including medical imaging, intelligent transportation, agriculture, and surveillance. Although there is substantial research potential for developing advanced algorithms for SSL-based semantic segmentation, a comprehensive study of existing methodologies is essential to trace advances and guide emerging researchers. This survey thoroughly investigates over 150 recent image segmentation articles, particularly focusing on SSL. It provides a practical categorization of pretext tasks, downstream tasks, and commonly used benchmark datasets for image segmentation research. It concludes with key observations distilled from a large body of literature and offers future directions to make this research field more accessible and comprehensible for readers.

Problem

• Reducing reliance on labeled data for image segmentation
• Surveying SSL methods for semantic segmentation tasks
• Categorizing pretext tasks and benchmark datasets

Innovation

• Uses self-supervised learning for image segmentation
• Exploits unlabeled data with surrogate tasks
• Surveys 150+ SSL-based segmentation methodologies