🤖 AI Summary
This study investigates how to achieve valid statistical inference when combining machine learning predictions with a small number of gold-standard labels, and clarifies its connections to classical survey sampling methods. Through theoretical analysis, it establishes for the first time the algebraic equivalence between the core estimator in prediction-powered inference (PPI) and model-assisted estimators from the 1970s—such as difference and generalized regression (GREG) estimators. The work systematically compares these approaches in terms of inferential paradigms, use of unlabeled data, and subgroup estimation error, delineating which aspects of PPI are inherited versus novel. It further proposes directions for integrating insights from both fields. These results ground PPI in classical survey sampling theory while simultaneously expanding the toolkit available for modern, nonstandard estimators within the survey sampling framework.
📝 Abstract
Prediction-powered inference (PPI) is a rapidly growing framework for combining machine learning predictions with a small set of gold-standard labels to conduct valid statistical inference. In this article, I argue that the core estimators underlying PPI are equivalent to well-established estimators from the survey sampling literature dating back to the 1970s. Specifically, the PPI estimator for a population mean is algebraically equivalent to the difference estimator of Cassel et al. (1976), and PPI plus corresponds to the generalized regression (GREG) estimator of Sarndal et al. (2003). Recognizing this equivalence, I consider what part of PPI is inherited from a long-standing literature in statistics, what part is genuinely new, and where inferential claims require care. After introducing the two frameworks and establishing their equivalence, I break down where PPI diverges from model-assisted estimation, including differences in the mode of inference, the role of the unlabeled data pool, and the consequences of differential prediction error for subgroup estimands such as the average treatment effect. I then identify what each framework offers the other: PPI researchers can draw on the survey sampling literature's well-developed theory of calibration, optimal allocation, and design-based diagnostics, while survey sampling researchers can benefit from PPI's extensions to non-standard estimands and its accessible software ecosystem. The article closes with a call for integration between these two communities, motivated by the growing use of large language models as measurement instruments in applied research.