Amortized Variational Inference for Logistic Regression with Missing Covariates

2026-03-22

AI Summary

This work addresses the inferential bias and degraded classification performance in logistic regression caused by missing covariates, particularly under high missingness rates and non-random missing mechanisms. The authors propose the AV-LR framework, which for the first time applies amortized variational inference directly to logistic regression with missing covariates. By jointly training a generative model, an amortized inference network, and a linear classification layer in an end-to-end fashion, AV-LR optimizes the evidence lower bound (ELBO) directly in the space of incomplete data—without introducing auxiliary latent variables—thereby simultaneously estimating regression parameters and the missingness mechanism. This approach avoids complex latent variable structures, achieves estimation accuracy comparable to or better than traditional EM algorithms on both synthetic and real-world datasets, incurs substantially lower computational cost, and demonstrates robust performance across diverse missing-data scenarios.

Technology Category

• Machine Learning: Statistical Relational/Logic Learning
• Computer Vision: Learning & Optimization for CV
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Missing covariate data pose a significant challenge to statistical inference and machine learning, particularly for classification tasks like logistic regression. Classical iterative approaches (EM, multiple imputation) are often computationally intensive, sensitive to high missingness rates, and limited in uncertainty propagation. Recent deep generative models based on VAEs show promise but rely on complex latent representations. We propose Amortized Variational Inference for Logistic Regression (AV-LR), a unified end-to-end framework for binary logistic regression with missing covariates. AV-LR integrates a probabilistic generative model with a simple amortized inference network, trained jointly by maximizing the evidence lower bound. Unlike competing methods, AV-LR performs inference directly in the space of missing data without additional latent variables, using a single inference network and a linear layer that jointly estimate regression parameters and the missingness mechanism. AV-LR achieves estimation accuracy comparable to or better than state-of-the-art EM-like algorithms, with significantly lower computational cost. It naturally extends to missing-not-at-random settings by explicitly modeling the missingness mechanism. Empirical results on synthetic and real-world datasets confirm its effectiveness and efficiency across various missing-data scenarios.

Problem

• missing covariates
• logistic regression
• statistical inference
• missing data
• binary classification

Innovation

• amortized variational inference
• logistic regression
• missing covariates
• evidence lower bound
• missing-not-at-random