Locally Convex Global Loss Network for Decision-Focused Learning

2024-03-04

AI Summary

Existing decision-making approaches under uncertainty suffer from a decoupled prediction-and-optimization pipeline and lack generalizability across arbitrary optimization structures. Method: This paper proposes an end-to-end Decision-Forward Learning (DFL) framework that directly optimizes for final decision quality. Its core innovation is the Locally Convex Global Loss Network (LCGLN), built upon Partially Input-Convex Neural Networks (PICNNs) to ensure gradient differentiability while preserving expressiveness for globally non-convex decision structures; it further integrates implicit differentiation with task-specific loss surrogates, eliminating reliance on explicit problem assumptions (e.g., convexity or closed-form solutions). Contribution/Results: Evaluated on three stochastic decision-making tasks, DFL achieves significant improvements in decision performance and stable convergence. It is compatible with any differentiable optimization layer, establishing the first generic, assumption-free, decision-centric learning paradigm.

Technology Category

• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

In decision-making problems under uncertainty, predicting unknown parameters is often considered independent of the optimization part. Decision-focused learning (DFL) is a task-oriented framework that integrates prediction and optimization by adapting the predictive model to give better decisions for the corresponding task. Here, an inevitable challenge arises when computing the gradients of the optimal decision with respect to the parameters. Existing research copes with this issue by smoothly reforming surrogate optimization or constructing surrogate loss functions that mimic task loss. However, they are applied to restricted optimization domains. In this paper, we propose Locally Convex Global Loss Network (LCGLN), a global surrogate loss model that can be implemented in a general DFL paradigm. LCGLN learns task loss via a partial input convex neural network which is guaranteed to be convex for chosen inputs while keeping the non-convex global structure for the other inputs. This enables LCGLN to admit general DFL through only a single surrogate loss without any sense for choosing appropriate parametric forms. We confirm the effectiveness and flexibility of LCGLN by evaluating our proposed model with three stochastic decision-making problems.

Problem

• Uncertainty Decision Making
• Decision-Focused Learning (DFL)
• Parameter-Agnostic Method

Innovation

• LCGLN
• Decision-Focused Learning
• Convex-Nonconvex Loss