🤖 AI Summary
To address the insufficient utilization of information in traditional two-stage imputation-prediction paradigms under Missing-Not-At-Random (MNAR) mechanisms, this paper proposes a two-stage adaptive optimization framework: it formulates prediction as a dynamically parameterized process driven by observed feature subsets, enabling joint implicit optimization of imputation and regression. The key contribution is the first establishment of an adaptive optimization theoretical framework for missing-data prediction, which rigorously uncovers the intrinsic equivalence between imputation and regression under joint learning, and extends this paradigm to nonlinear models. Extensive experiments demonstrate that, under strong MNAR settings, our method improves test accuracy by 2–10% over classical two-stage approaches, while significantly enhancing generalization capability.
📝 Abstract
When training predictive models on data with missing entries, the most widely used and versatile approach is a pipeline technique where we first impute missing entries and then compute predictions. In this paper, we view prediction with missing data as a two-stage adaptive optimization problem and propose a new class of models, adaptive linear regression models, where the regression coefficients adapt to the set of observed features. We show that some adaptive linear regression models are equivalent to learning an imputation rule and a downstream linear regression model simultaneously instead of sequentially. We leverage this joint-impute-then-regress interpretation to generalize our framework to non-linear models. In settings where data is strongly not missing at random, our methods achieve a 2-10% improvement in out-of-sample accuracy.