Informative missingness and its implications in semi-supervised learning

2025-12-03

AI Summary

This work addresses semi-supervised classification under informative label missingness—where the missingness mechanism depends on features or class labels. To overcome the limitation of conventional semi-supervised learning (SSL) methods, which ignore information embedded in the missingness mechanism, we propose a likelihood-based framework grounded in finite mixture models that explicitly models the label missingness process and jointly optimizes labeled and unlabeled data via the EM algorithm. Theoretical analysis shows that, under informative missingness, our approach achieves lower asymptotic expected error than supervised learning using only fully labeled data. Empirical results demonstrate substantial accuracy gains in settings with moderate class overlap and sparse labeling. To our knowledge, this is the first work to systematically reveal and exploit the statistical information carried by the missingness indicator within a unified likelihood inference framework, thereby bridging statistical missing-data modeling and semi-supervised learning.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Semi-supervised learning (SSL) constructs classifiers using both labelled and unlabelled data. It leverages information from labelled samples, whose acquisition is often costly or labour-intensive, together with unlabelled data to enhance prediction performance. This defines an incomplete-data problem, which statistically can be formulated within the likelihood framework for finite mixture models that can be fitted using the expectation-maximisation (EM) algorithm. Ideally, one would prefer a completely labelled sample, as one would anticipate that a labelled observation provides more information than an unlabelled one. However, when the mechanism governing label absence depends on the observed features or the class labels or both, the missingness indicators themselves contain useful information. In certain situations, the information gained from modelling the missing-label mechanism can even outweigh the loss due to missing labels, yielding a classifier with a smaller expected error than one based on a completely labelled sample analysed. This improvement arises particularly when class overlap is moderate, labelled data are sparse, and the missingness is informative. Modelling such informative missingness thus offers a coherent statistical framework that unifies likelihood-based inference with the behaviour of empirical SSL methods.

Problem

• Addresses informative missingness in semi-supervised learning classification
• Examines when missing labels provide more information than complete labels
• Unifies likelihood-based inference with empirical SSL methods behavior

Innovation

• Modeling informative missingness in SSL
• Using EM algorithm for likelihood framework
• Incorporating missing-label mechanism information