Adaptive Margin Contrastive Learning for Ambiguity-aware 3D Semantic Segmentation

2024-07-15

IEEE International Conference on Multimedia and Expo

AI Summary

In 3D point cloud semantic segmentation, ambiguous labels in transitional regions and unreliable manual annotations degrade model performance. To address this, we propose AMContrast3D, an adaptive margin contrastive learning framework. Its core innovation is the first introduction of a fuzziness-aware dynamic margin mechanism: point-wise fuzziness is estimated via positional embeddings, and the contrastive learning margin—allowing for negative values—is adaptively adjusted accordingly, relaxing constraints for highly ambiguous points while strengthening discrimination for confident ones. Crucially, AMContrast3D requires no additional annotations. Evaluated on S3DIS and ScanNet, it significantly improves robustness in boundary regions and achieves state-of-the-art overall mIoU.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

In this paper, we propose an adaptive margin contrastive learning method for 3D point cloud semantic segmentation, namely AMContrast3D. Most existing methods use equally penalized objectives, which ignore per-point ambiguities and less discriminated features stemming from transition regions. However, as highly ambiguous points may be indistinguishable even for humans, their manually annotated labels are less reliable, and hard constraints over these points would lead to sub-optimal models. To address this, we design adaptive objectives for individual points based on their ambiguity levels, aiming to ensure the correctness of low-ambiguity points while allowing mistakes for high-ambiguity points. Specifically, we first estimate ambiguities based on position embeddings. Then, we develop a margin generator to shift decision boundaries for contrastive feature embeddings, so margins are narrowed due to increasing ambiguities with even negative margins for extremely high-ambiguity points. Experimental results on large-scale datasets, S3DIS and ScanNet, demonstrate that our method outperforms state-of-the-art methods.

Problem

• Adaptive margin for 3D segmentation
• Handling ambiguous points in clouds
• Improving feature discrimination accuracy

Innovation

• Adaptive margin contrastive learning
• Ambiguity-aware semantic segmentation
• Dynamic margin generator for features