🤖 AI Summary
This study addresses the limitation of existing machine learning methods, which prioritize predictive accuracy while neglecting design-unbiasedness—a critical requirement in official statistics and similar domains. The authors propose a general framework that does not rely on assumptions about the true data-generating model and, for the first time, integrates the known inclusion mechanisms from probability sampling designs into every stage of the learning pipeline: training sample selection, hyperparameter tuning, and performance evaluation. This integration guarantees design-unbiased prediction and classification over finite populations. The approach is compatible with popular algorithms such as k-nearest neighbors and random forests, establishes theoretical conditions under which design-unbiasedness is achieved, and provides practical algorithmic implementations alongside evaluation criteria.
📝 Abstract
Machine Learning (ML) algorithms, such as k-Nearest Neighbours (kNN) or random forest, eschew the ideal of true data models in favour of predictive performance. However, minimising the MSE or F-score cannot lead to unbiasedness directly, which is important in many situations such as official statistics. We study the conditions of algorithmic ML, other than the existence and knowledge of true data models, which lead to unbiased prediction or classification for a given finite population, including how the training data may be sampled from the population, how a trained prediction algorithm can be tuned to achieve unbiased prediction or classification for that population, and how the performance of out-of-sample prediction or classification can be assessed unbiasedly. The inference is based on the known probability design of samples and training sets, rather than any assumed distributions or models.