Deep Fréchet Regression

2024-07-31

arXiv.org

AI Summary

This paper addresses regression modeling between non-Euclidean response variables—such as probability distributions and networks—and high-dimensional Euclidean predictors, overcoming limitations of traditional linear and parametric assumptions. It tackles the dual challenges of the curse of dimensionality in nonparametric regression and the absence of linear structure in metric spaces. We propose the first inverse-mapping framework integrating deep neural networks with local Fréchet regression, pioneering the incorporation of manifold learning into Fréchet regression. We establish a convergence theory accommodating measurement error in predictors. Under sub-Gaussian noise, we derive generalization error bounds and prove that the theoretical convergence rate achieves the optimal minimax order. Extensive simulations and real-data analyses demonstrate that our method significantly outperforms existing approaches on distributional and network-valued data, while maintaining robustness and high predictive accuracy.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Advancements in modern science have led to the increasing availability of non-Euclidean data in metric spaces. This paper addresses the challenge of modeling relationships between non-Euclidean responses and multivariate Euclidean predictors. We propose a flexible regression model capable of handling high-dimensional predictors without imposing parametric assumptions. Two primary challenges are addressed: the curse of dimensionality in nonparametric regression and the absence of linear structure in general metric spaces. The former is tackled using deep neural networks, while for the latter we demonstrate the feasibility of mapping the metric space where responses reside to a low-dimensional Euclidean space using manifold learning. We introduce a reverse mapping approach, employing local Fr'echet regression, to map the low-dimensional manifold representations back to objects in the original metric space. We develop a theoretical framework, investigating the convergence rate of deep neural networks under dependent sub-Gaussian noise with bias. The convergence rate of the proposed regression model is then obtained by expanding the scope of local Fr'echet regression to accommodate multivariate predictors in the presence of errors in predictors. Simulations and case studies show that the proposed model outperforms existing methods for non-Euclidean responses, focusing on the special cases of probability distributions and networks.

Problem

• Modeling relationships between non-Euclidean responses and Euclidean predictors
• Handling high-dimensional predictors without parametric assumptions
• Mapping metric space responses to low-dimensional Euclidean representations

Innovation

• Deep neural networks handle high-dimensional predictors
• Manifold learning maps metric space to Euclidean space
• Local Fréchet regression reverses mapping to original space