Sufficiently Reduced Distributional Regression

2026-09-24

AI Summary

This study addresses the challenge of jointly performing conditional distribution estimation and nonlinear sufficient dimension reduction in high-dimensional data by proposing the SRDR method. SRDR reformulates reduction sufficiency as a risk minimization problem grounded in strictly proper scoring rules. Specifically, it jointly trains a nonlinear dimension reduction mapping and a generative model through energy score minimization, thereby circumventing adversarial training and explicit density computation. The asymptotic sufficiency of the learned representations is established theoretically. Experimental results demonstrate that SRDR effectively recovers low-dimensional structures in tasks such as CT localization, achieving predictive performance that matches or surpasses existing nonlinear sufficient dimension reduction methods.

Abstract

We propose Sufficiently Reduced Distributional Regression (SRDR), a generative method that combines conditional distribution estimation with nonlinear sufficient dimension reduction (SDR). It builds on a characterization of sufficiency through strictly proper scoring rules: a dimension reduction is sufficient if and only if predicting the response from the reduced covariates incurs no loss in expected score relative to the full covariates. Sufficient dimension reduction thus becomes a risk minimization problem. SRDR jointly trains a dimension reduction map and a generative prediction model by minimizing the energy score, which can be estimated by sampling without density evaluation or adversarial training. The framework extends to multi-environment data and to classification. We prove that the estimated conditional distributions converge in energy distance to the true ones, which implies that the learned representation is asymptotically sufficient. In simulations and applications to CT slice localization, superconductivity, and digit classification, SRDR recovers low-dimensional sufficient structure and matches or outperforms state-of-the-art nonlinear SDR methods in representation quality and predictive performance.

Problem

• Sufficient Dimension Reduction
• Conditional Distribution Estimation
• Distributional Regression
• Nonlinear Dimensionality Reduction
• Proper Scoring Rules

Innovation

• Sufficient Dimension Reduction
• Distributional Regression
• Energy Score
• Strictly Proper Scoring Rules
• Generative Model