🤖 AI Summary
In decision-focused learning (DFL), the gradients of predicted parameters—derived from optimization problems such as linear programming—are almost everywhere zero, causing gradient vanishing when directly optimizing decision regret. Existing smoothing or surrogate loss approaches fail to resolve this fundamental issue. This paper proposes a surrogate-loss-driven end-to-end framework: even when employing differentiable optimization layers (e.g., DYS-Net), we explicitly minimize a carefully designed surrogate loss to ensure consistent gradient flow and alignment with decision quality. Theoretically, we prove that smoothing cannot eliminate the inherent zero-measure set of non-differentiability, whereas our surrogate loss circumvents this bottleneck entirely. Experiments demonstrate that our method achieves state-of-the-art or superior decision regret across benchmarks, while accelerating training significantly—delivering both higher accuracy and computational efficiency.
📝 Abstract
Decision-focused learning (DFL) trains a machine learning (ML) model to predict parameters of an optimization problem, to directly minimize decision regret, i.e., maximize decision quality. Gradient-based DFL requires computing the derivative of the solution to the optimization problem with respect to the predicted parameters. However, for many optimization problems, such as linear programs (LPs), the gradient of the regret with respect to the predicted parameters is zero almost everywhere. Existing gradient-based DFL approaches for LPs try to circumvent this issue in one of two ways: (a) smoothing the LP into a differentiable optimization problem by adding a quadratic regularizer and then minimizing the regret directly or (b) minimizing surrogate losses that have informative (sub)gradients. In this paper, we show that the former approach still results in zero gradients, because even after smoothing the regret remains constant across large regions of the parameter space. To address this, we propose minimizing surrogate losses -- even when a differentiable optimization layer is used and regret can be minimized directly. Our experiments demonstrate that minimizing surrogate losses allows differentiable optimization layers to achieve regret comparable to or better than surrogate-loss based DFL methods. Further, we demonstrate that this also holds for DYS-Net, a recently proposed differentiable optimization technique for LPs, that computes approximate solutions and gradients through operations that can be performed using feedforward neural network layers. Because DYS-Net executes the forward and the backward pass very efficiently, by minimizing surrogate losses using DYS-Net, we are able to attain regret on par with the state-of-the-art while reducing training time by a significant margin.