Uncertainty and Energy based Loss Guided Semi-Supervised Semantic Segmentation

2025-01-03

AI Summary

To address the challenges of high noise in pseudo-labels and severe class distribution shift in semi-supervised semantic segmentation, this paper proposes a novel framework integrating data uncertainty modeling with energy-based generative learning. Methodologically, it is the first to jointly incorporate pixel-wise heteroscedastic aleatoric uncertainty estimation and an energy-function-driven loss into semi-supervised segmentation, enabling pseudo-label quality awareness and discriminative-generative co-optimization. A dual-branch network architecture is designed, combining pseudo-intersection/pseudo-union labeling strategies with consistency regularization. Extensive experiments demonstrate state-of-the-art performance: on PASCAL VOC and Cityscapes, the proposed method achieves absolute mIoU improvements of 2.3–3.1 percentage points over prior works. Notably, it exhibits significantly enhanced robustness and generalization under extremely low labeling ratios (≤10%), validating its effectiveness in data-scarce regimes.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Segmentation
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Semi-supervised (SS) semantic segmentation exploits both labeled and unlabeled images to overcome tedious and costly pixel-level annotation problems. Pseudolabel supervision is one of the core approaches of training networks with both pseudo labels and ground-truth labels. This work uses aleatoric or data uncertainty and energy based modeling in intersection-union pseudo supervised network.The aleatoric uncertainty is modeling the inherent noise variations of the data in a network with two predictive branches. The per-pixel variance parameter obtained from the network gives a quantitative idea about the data uncertainty. Moreover, energy-based loss realizes the potential of generative modeling on the downstream SS segmentation task. The aleatoric and energy loss are applied in conjunction with pseudo-intersection labels, pseudo-union labels, and ground-truth on the respective network branch. The comparative analysis with state-of-the-art methods has shown improvement in performance metrics.

Problem

• Computer Vision
• Object Recognition
• Limited Annotated Data

Innovation

• Random Uncertainty Integration
• Energy-based Modeling
• Semi-supervised Semantic Segmentation