Pessimistic bilevel optimization approach for decision-focused learning

2025-01-28

AI Summary

This work addresses contextual combinatorial optimization under stochastic uncertainty, aiming to reduce decision regret induced by prediction errors—particularly improving generalization to unseen instances. We propose the first decision-focused learning framework grounded in pessimistic bilevel optimization, which explicitly embeds combinatorial decision structure into the predictive modeling process and directly minimizes worst-case decision loss, thereby circumventing error propagation inherent in conventional two-stage approaches. Key innovations include: (i) an ε-approximate bilevel optimization formulation; (ii) a tailored cutting-plane algorithm for efficient computation; and (iii) decision-focused gradient-based implicit differentiation. Evaluated on the 0–1 knapsack problem, our method reduces out-of-sample regret by 23.7% on average over baselines including SPO+, while demonstrating superior robustness and generalization across diverse problem instances.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Combinatorial Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The recent interest in contextual optimization problems, where randomness is associated with side information, has led to two primary strategies for formulation and solution. The first, estimate-then-optimize, separates the estimation of the problem's parameters from the optimization process. The second, decision-focused optimization, integrates the optimization problem's structure directly into the prediction procedure. In this work, we propose a pessimistic bilevel approach for solving general decision-focused formulations of combinatorial optimization problems. Our method solves an $varepsilon$-approximation of the pessimistic bilevel problem using a specialized cut generation algorithm. We benchmark its performance on the 0-1 knapsack problem against estimate-then-optimize and decision-focused methods, including the popular SPO+ approach. Computational experiments highlight the proposed method's advantages, particularly in reducing out-of-sample regret.

Problem

• Regret Minimization
• Combinatorial Optimization
• Contextual Information

Innovation

• Combination Optimization
• Decision-making under Contextual Uncertainty
• Reduction of Decision Regret