On the Nonconvexity of Push-Forward Constraints and Its Consequences in Machine Learning

2024-03-12

SIAM Journal on Mathematics of Data Science

AI Summary

This work addresses the (non)convexity of push-forward constraints in machine learning, exposing their fundamental role in impeding convex reformulations of optimization problems arising in optimal transport, generative modeling, and algorithmic fairness. Method: Integrating measure theory, convex analysis, and optimal transport theory, we derive necessary and sufficient conditions for the convexity of the set of push-forward mappings. Contribution/Results: We rigorously prove that, except in degenerate cases, push-forward constraints are intrinsically nonconvex for almost all probability measures. This reveals a foundational theoretical limitation of existing convex optimization frameworks relying on push-forward constraints—such as fair predictors and Wasserstein generative models. Beyond constructing explicit counterexamples and geometric characterizations, we uncover the functional-space structural origins of this nonconvexity. Our results provide critical theoretical guidance and motivate new directions for designing robust learning algorithms resilient to such inherent nonconvexity.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Security and privacy of machine learning and AI applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The push-forward operation enables one to redistribute a probability measure through a deterministic map. It plays a key role in statistics and optimization: many learning problems (notably from optimal transport, generative modeling, and algorithmic fairness) include constraints or penalties framed as push-forward conditions on the model. However, the literature lacks general theoretical insights on the (non)convexity of such constraints and its consequences on the associated learning problems. This paper aims at filling this gap. In the first part, we provide a range of sufficient and necessary conditions for the (non)convexity of two sets of functions: the maps transporting one probability measure to another and the maps inducing equal output distributions across distinct probability measures. This highlights that for most probability measures, these push-forward constraints are not convex. In the second part, we show how this result implies critical limitations on the design of convex optimization problems for learning generative models or groupwise fair predictors. This work will hopefully help researchers and practitioners have a better understanding of the critical impact of push-forward conditions onto convexity.

Problem

• Analyzing nonconvexity of push-forward constraints in learning problems
• Exploring impact of nonconvexity on generative models and fairness
• Providing theoretical insights for push-forward conditions in optimization

Innovation

• Analyzes nonconvexity of push-forward constraints in learning
• Provides conditions for convexity in probability measure transport
• Highlights limitations in convex optimization for fair predictors