Incomplete Multi-view Multi-label Classification via a Dual-level Contrastive Learning Framework

2024-11-27

arXiv.org

AI Summary

To address the practical challenge of simultaneous view and label missingness in multi-view multi-label classification, this paper proposes a dual-level contrastive learning framework. Methodologically, we design a dual-channel disentanglement module to separate shared consensus representations from view-specific representations. We further formulate a cross-level contrastive objective: at the feature level, we employ InfoNCE loss; at the semantic level, we introduce label-level semantic contrast—marking the first approach to fully disentangle heterogeneous view and label information under dual missingness. By jointly optimizing shared and view-specific representations, our model achieves significant improvements over state-of-the-art methods across multiple benchmark datasets, with mean average precision (mAP) gains of 3.2–5.8 percentage points. Moreover, the model demonstrates enhanced robustness and stability in multi-label classification performance.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recently, multi-view and multi-label classification have become significant domains for comprehensive data analysis and exploration. However, incompleteness both in views and labels is still a real-world scenario for multi-view multi-label classification. In this paper, we seek to focus on double missing multi-view multi-label classification tasks and propose our dual-level contrastive learning framework to solve this issue. Different from the existing works, which couple consistent information and view-specific information in the same feature space, we decouple the two heterogeneous properties into different spaces and employ contrastive learning theory to fully disentangle the two properties. Specifically, our method first introduces a two-channel decoupling module that contains a shared representation and a view-proprietary representation to effectively extract consistency and complementarity information across all views. Second, to efficiently filter out high-quality consistent information from multi-view representations, two consistency objectives based on contrastive learning are conducted on the high-level features and the semantic labels, respectively. Extensive experiments on several widely used benchmark datasets demonstrate that the proposed method has more stable and superior classification performance.

Problem

• Addresses incomplete multi-view multi-label classification with missing views and labels
• Decouples consistent and view-specific information into separate feature spaces
• Uses dual-level contrastive learning to extract high-quality consistent information

Innovation

• Decouples consistent and view-specific information into different spaces
• Uses two-channel module for shared and proprietary representations
• Applies contrastive learning on high-level features and semantic labels