Rectified Linear Unit Regression

2026-05-28

AI Summary

This study addresses the challenge of identifying and inferring integral functionals of conditional distributions with discontinuous outcomes by proposing a novel ReLU regression framework. The approach constructs closed-form estimators through covariate projection of ReLU-transformed outcome variables and recovers integrated conditional quantile functions via their convex conjugates obtained through the Legendre–Fenchel transform. By introducing ReLU activation into regression, this method enables direct identification of conditional distribution features under only weak distributional assumptions, accommodates discontinuous outcomes, and identifies average quantile treatment effects over arbitrary probability intervals. Leveraging Hadamard directional differentiability and the Delta method, the authors establish a unified theory for the consistent asymptotic distribution of the proposed estimators, substantially expanding the set of distributional parameters identifiable in empirical research.

Abstract

This paper develops a regression framework for the direct estimation of integrated functionals of conditional outcome distributions. The proposed method, termed rectified linear unit (ReLU) regression, projects the ReLU-transformed outcome onto covariates and admits a closed-form estimator. Its population regression function coincides with the integrated conditional distribution function of the outcome, and its convex conjugate, obtained via the Legendre-Fenchel transformation, recovers the integrated conditional quantile function. Both the regression and its conjugate require only mild distributional assumptions and accommodate non-continuous outcomes. We establish the uniform asymptotic distribution of the estimator and develop inference for the conjugate functional via the delta method for Hadamard directionally differentiable maps. Building on these results, we establish identification and inference for average quantile treatment effects over arbitrary subintervals of probability levels. This broadens the set of distributional parameters available to empirical work.

Problem

• integrated functionals
• conditional distribution
• quantile treatment effects
• non-continuous outcomes
• distributional inference

Innovation

• ReLU regression
• integrated conditional distribution
• convex conjugate
• average quantile treatment effects
• Hadamard directional differentiability