Scaling Decision-Focused Learning to Large Problems with Lagrangian Decomposition

2026-06-07

AI Summary

This work addresses the limitations of traditional decision-focused learning in large-scale predict-then-optimize problems, which suffer from high computational costs and poor scalability. The authors introduce Lagrangian decomposition into this framework for the first time, designing a novel surrogate objective and associated loss function. They further propose two variants that effectively balance efficiency and accuracy, seamlessly integrating with mainstream algorithms such as SPO+ and IMLE. The resulting approach significantly enhances scalability and parallelizability on large-scale instances, successfully handling problems with up to eight times more variables than previously feasible in multidimensional knapsack and quadratic portfolio optimization tasks. Empirical results demonstrate superior performance over existing methods while maintaining strong parallel efficiency.

Abstract

Decision-focused learning has shown great promise for addressing predict-then-optimize problems, particularly in the presence of under-specified models. However, its practical deployment is often hindered by high computational costs and limited scalability, as it requires solving a constrained optimization problem for each training instance at every iteration. To address these challenges, we propose a novel framework that incorporates Lagrangian decomposition into the decision-focused learning paradigm. Specifically, we introduce a new surrogate objective along with two loss functions for evaluating and training the underlying prediction model. We further propose two variants of our approach, which offer different trade-offs between computational efficiency and solution quality. Our framework can be seamlessly integrated with standard decision-focused learning methods, including Smart Predict-then-Optimize (SPO+) and Implicit Maximum Likelihood Estimation (IMLE). Through experiments on two standard benchmarks, the multi-dimensional knapsack problem and quadratic portfolio optimization, we demonstrate that our approach achieves competitive performance while remaining amenable to parallelization. In particular, it consistently outperforms traditional decision-focused learning methods on large-scale instances, involving up to eight times more variables than those typically considered in related work. The implementation is available at https://github.com/corail-research/DFL-LD.

Problem

• decision-focused learning
• scalability
• computational cost
• predict-then-optimize
• constrained optimization

Innovation

• Decision-Focused Learning
• Lagrangian Decomposition
• Scalability
• Surrogate Objective
• Parallelization