Generalized Engression Models

2026-10-01

AI Summary

This study addresses the challenge of conditional distribution estimation for mixed-type multivariate outcomes by proposing a generalized Engression model that establishes a unified nonparametric regression framework. By introducing data-specific link functions and stochastic perturbation smoothing, the method overcomes the non-differentiability bottleneck inherent in discontinuous link functions. This enables end-to-end joint distribution learning and gradient-based optimization across continuous, discrete, and ordinal outputs, supported by a general representation theorem. Evaluated on ecological and health benchmarks, the proposed approach achieves superior joint distribution accuracy compared to specialized models, while matching or exceeding state-of-the-art state-space models. These results validate the broad applicability and effectiveness of the generalized Engression framework for complex mixed-type outcome modeling.

Abstract

We consider estimating the conditional distribution of a multivariate outcome given covariates when its coordinates may be continuous, binary, categorical, ordinal or rankings, and are conditionally dependent on one another. Different statistical methods have been developed for each outcome type, and most of them target a summary of the conditional distribution, such as the mean of each coordinate, rather than the joint distribution of the outcome vector. We develop generalized engression models, a unified nonparametric distributional regression framework for outcomes of any type. The proposed method builds upon engression, a scoring-rule-based deep generative model, and introduces a data-type-specific link function and a stochastic perturbation that smooths the loss, enabling gradient-based training even with discontinuous links. We establish universal representation results for continuous, discrete and mixed outcomes. In simulations and in two applications, 242 species in a community ecology benchmark and a 17-dimensional mixed-type health outcome, the method matches type-specific models on marginal scores, improves on them on the joint distribution, and matches or exceeds purpose-built state-of-the-art joint species distribution models. Software is available in Python.

Problem

• conditional distribution estimation
• multivariate mixed-type outcomes
• distributional regression
• joint distribution modeling

Innovation

• Generalized Engression Models
• Nonparametric Distributional Regression
• Scoring-rule-based Deep Generative Model
• Stochastic Perturbation
• Mixed-type Outcomes